University of Bath

The university developed “Principles of research assessment and management” to formalize good practices of evaluation at the University of Bath.¹ While the Principles do not represent a new approach for the university, or even a break with the past, they articulate the existing campus culture in order to deepen and strengthen it.

To create the Principles, a diverse “task and finish” working group was created, chaired by the Pro Vice Chancellor for Research. Group members were purposeful in their design, looking for principles that could be applied broadly across disciplines. Moreover, the principles have a practical use in reinforcing cultural norms, in that faculty and staff can refer to them to challenge instances of bad practices

Development of the statement of principles was prompted by the Metric Tide (2015)² and the Leiden Manifesto (2015)³. The Principles establish “expert judgement” as the basis for research assessment:

Criteria and/or indicators used must be carefully chosen in light of the purpose of the assessment and context. Where appropriate, quantitative indicators can be used to inform judgements and challenge preconceptions, but not to replace expert judgement.

Recognizing there is not a one-size-fits-all approach to responsible research assessment, the Principles highlight the need for more tailored approaches, as “disciplinary differences in research inputs, processes and outputs have to be taken into account.” The Principles also explicitly state that the scale of the research activity must be considered, and “particular caution is needed when interpreting quantitative indicators in small scale assessments such as the assessment of an individual researcher.”

The task and finish working group included plans to review the Principles. So in 2018, a year after their release, a follow-up survey was conducted to assess the awareness and utility of the statement of Principles. The survey results highlighted good understanding of the relevance of the Principles and a need for continual promotion of the Principles. The group disbanded after achieving its goal of developing and distributing the Principles.

University of Bath is currently looking in more detail at assessment of academic staff and researchers.

The purpose of the Principles was to articulate the existing culture at the university.¹ However, the process also drew inspiration from external influences.

For example, one impetus for the statement of principles was the publication of the Metric Tide in 2015.² While this external event may have started the process, several other internal drivers, including the need to raise awareness and reinforce a responsible academic culture, was largely responsible for driving change. Similarly, the motivation for future and ongoing assessment reform is to consolidate and demonstrate the University’s commitment to good practice in research evaluation.

When discussing limitations, it was noted that while the broad openness of the statement of Principles “helped achieve [bottom-up] buy-in,” a significant remaining challenge is to develop specific guidelines that are flexible enough for academics in different disciplines and career stages. Another challenge is the concern for assessment portability and ensuring that the university’s researchers are not disadvantaged. In particular, there is a concern for early career researchers, who may not be evaluated the same way at other institutions.

In developing and distributing the Principles, there was a combined top-down/bottom-up dynamic.¹ The development of the Principles was initiated in 2015 by a partnership between the Library and the Research and Innovation Services staff, with senior leadership provided by the Pro Vice Chancellor for Research.

The University formed a working group in 2016, which was responsible for the development, initial implementation, and review of the “Principles of research assessment and management.” The Principles were deliberately developed with a broad acceptance in mind, and the staffing of the working group “was quite deliberate” in order to maintain interest and support from all levels and disciplines. The Principles had more credibility as a result of the initial buy-in from senior leadership.

Specific obstacles faced were: limited awareness of research assessment reform and its potential benefits; lack of evidence on potential benefits of research assessment reform; complexity of research assessment reform (e.g., different national and disciplinary practices); alignment of institutional assessment procedures with nationally and internationally dominant procedures; lack of institutional autonomy due to national/regional rules and regulations; and lack of institutional autonomy due to rules and regulations imposed by research funding organizations.

After the publication of Metric Tide in 2015,² the University agreed to set up a working group to develop institutional Principles.¹ In 2016 the working group drafted the statement of Principles and a distribution plan. The statement was approved by the Senate in February 2017 and published online in March 2017.

In July 2018, working group members reconvened to review the dissemination of the institution’s Principles. They also introduced explicit mechanisms for reporting concerns about activities not conforming to the Principles.

The next step is to review practices for the assessment of academic staff and researchers.