Case Study

Open University of Catalonia

Interview conducted 7 October, 2020  Compare case studies

The Open University of Catalonia (UOC) has been moving the focus of their assessment criteria and practices for recruitment and career progression away from journal-based outputs to a much broader discussion of achievements. Due to the centralization of career progression for faculty nationally (through agencies), the new assessment criteria apply only to postdoctoral fellows and UOC research staff and not for professorial hirings.

As the history and academic culture at UOC are deeply rooted in Open Knowledge, there were several internal drivers to push career progression and recruitment evaluations to promote open scholarship. In 2018, a DORA task force was created after initial advocacy by researchers and staff, that presented a report on how to approach a research evaluation reform in UOC. In January 2019, UOC became the first Spanish university (as a whole) to sign the San Francisco Declaration on Research Assessment (DORA) and created the DORA Action Plan. Following that, the Open Knowledge Action Plan was published in mid-2019, and that defined the main areas of work for responsibly restructuring the research assessment models at UOC. In the UOC Strategic Plan of the University 2022-2025, a specific progress path in research assessment was determined, which helped the institution to take the decision of signing the Coalition for Advancing Research Assessment (CoARA).

Interview originally conducted October 2020. Case study updated August 2023.

Who: Organization profile

Country Catalonia (Spain)
Profile of institution distance learning university
Number of FTE researchers 500-1,000
Further information UOC signs the San Francisco Declaration (DORA) Action Plan
Organization of research evaluation Institutional/university levelresearch unit levels
Who is involved? academic leadershipacademic researchers

What: What changed and the key elements of change

After the publication of the DORA Action Plan, the university has been working toward changing its assessment criteria for career progression and recruitment practices. Faculty recruitment and career progression align with the three central university missions: teaching, research, and the so-called “third mission,” which is related to societal interactions and impacts.

The change from journal-based metrics to a narrative discussion of “achievements” was implemented for postdoctoral recruitment, including papers, research projects, patents and more. Under the new criteria, “increasing importance is given to research content and its social impact, rather than just its appearance in various journals and metrics.” With the changes proposed within the Action Plan, UOC aims to transform “evaluation methods using more qualitative, transparent, fair, inclusive and socially relevant formulas that take into consideration not only research quality, but also the societal impact of our research.”

For the recruitment of research group leaders (full-time researchers at UOC who are not professors), a specific international Scientific Advisory Board analyzes the applicant’s CV and research plan by means of a written proposal and a public presentation. Assessment for the career progression of research group leaders takes place every four years with the advisory board producing a qualitative evaluation report. Professors are assessed by the Dean, Vice-Rector for Teaching, Vice-Rector for Research as well as the Rector in case of promotion to full professor along with two or three other professors. Assessment is peer-reviewed and only responsible use of metrics is allowed.

At the end of 2022, different universities and research institutions around Europe built and signed the Agreement on Reforming Research Assessment which established a common direction for the research assessment reforms. Institutions that sign CoARA are required to develop an individual five-year action plan. UOC is one of the member organizations of CoARA and has been working on building the Action Plan for 2024 to 2027. The UOC's long-term objective will be to remodel its research assessment system to align with the European reform proposal, necessarily encouraging quality research, considering the diversity of processes and outcomes, facilitating the diversity of academic profiles, avoiding any discrimination in the current system, but adapting to the specific characteristics of UOC and the national and regional context.

Why: Motivation for change

The history and academic culture at UOC are rooted in Open Science. As such, there were several internal drivers to push career progression and recruitment evaluations to be more open. Identifying ways to recognize and reward the social impact of research was also a major motivation for change. “Much of the impact of research cannot be explained or assessed with quantitative indicators, instead it requires a report.”

After initial advocacy by researchers and staff at UOC, the Research and Innovation Committee, chaired by the Vice President for Research, created a DORA task force. This combined bottom-up/top-down approach facilitated the necessary dialogue and advocacy in linking open science with research assessment reform. The DORA Action Plan created by the task force was part of the wider Open Knowledge Access Plan at UOC.

The task force was pivotal for capacity building, in terms of both defining achievements and for designing rubrics to guide evaluators of internal calls. It was also critical for advocacy and community engagement to help explain the new process and ensure that evaluation would be consistent, with a focus on impact and peer-reviewed evaluation, and with responsible use of journal-based metrics.

The main obstacle faced was the difference between internal and external requirements, defined by external actors, as well as the alignment of institutional assessment procedures with nationally and internationally dominant procedures. In 2023, UOC is hopeful of new opportunities that will arise from signing CoARA and through being part of the community being developed from CoARA. UOC expects that in the following years they can complete their transition to a new research assessment model whose process began with signing DORA.

How: Processes and dynamics for developing, implementing and managing change

The DORA task force was created by the Research and Innovation Committee in early 2018. The task force presented the “UOC signs the San Francisco Declaration (DORA) Action Plan” in December 2018.

In 2021, it was approved by the Executive Board and published, the UOC Open Knowledge Policy, where it stated the compromise to include open knowledge principles in internal assessment processes and in all institutional actions and policies in accordance with the DORA principles.

Upon signing the Agreement, the UOC commits to formulating an action plan for 2024 to 2027 within a year. This action plan will be uniquely tailored factoring in the University's current status alongside the prevailing national and regional context.

When: Timeline for development and implementation

The DORA task force was created by the Research and Innovation Committee in early 2018. The task force presented the “UOC signs the San Francisco Declaration (DORA) Action Plan” in December 2018. The progress on action items for open science and assessment is currently ongoing.


See hyperlinks in the text above.