Universities Norway

In the fall of 2019, Universities Norway established a research assessment working group to articulate and improve existing practices through building “a framework for how to think about career assessment in light of open science.” Their framework, which is set to release by the end of 2020, will provide structured career assessment guidelines, modeled in part after the European Commission’s Open Science Career Assessment Matrix (OS-CAM).¹

The proposed Norwegian Career Assessment Matrix (NOR-CAM) draws inspiration from OS-CAM but is adapted for the Norwegian context and has been further developed into a generic framework that includes but goes beyond open science. It uses similar categories to those in OS-CAM: research output, research process, educational competency, leadership and service, societal interaction and impact, and other experience. In addition to these six categories, which form the rows in the Norwegian assessment matrix, there are three columns:

• Description of criteria with clear definitions
• Documentation and objective evidence for each criterion
• Reflection and qualitative self-assessment

In Norway, the structure for hiring and career progression is relatively uniform and framed by government regulations. Importantly, the law is sufficiently open-ended, and recruitment and hiring practices have gradually shifted over the past decade from a narrow focus on publications-based proxy measures toward a more holistic view of researcher accomplishments. However, significant variations between institutions and academic fields still remain.

Current practices are detailed, well documented, and increasingly holistic, often including a focus on narrative self-evaluation. But it has still been necessary to issue recommendations to use publication metrics with caution and not as a basis for decisions on an individual level. The anticipated report from the Universities Norway working group is expected to communicate and extend current good practices across Norway by articulating a more systematic and structured approach to assessment.

Internal drivers within Norway, such as the transition to Open Science and support for DORA, established a need for action and inspired the working group. However, the formation of the working group was sparked by external forces, such as the national assessment reform efforts in Finland and the Netherlands, as well as the European University Association’s Expert Group on Science 2.0/Open Science.

Because of a desire to create a Norwegian system “that will work well [also] in an international context,” the changes within Europe were critical to establishing the working group. Stakeholders were keen to implement national assessment reform within an international context.

A major goal of the working group is to improve and develop the current assessment system. The “adoption of the same system and approach to assessment will greatly simplify the process for everyone.” The current criteria for universities in Norway, as established by the government, is relatively detailed and holistic; what is lacking is a “systematic approach to collect information and [consensus] in how to present [the information] in an easy way.” The long-term goal of the working group is to develop tools and build capacity for the national documentation matrix. For example, the working group is focused on building digital tools to make currently available material more integrated and accessible.

Bottom-up change was stimulated by individual academic stakeholders, including working group members. This approach to reform is within the larger context of a parallel government-initiated working group on career structures at large, with a focus on education-driven careers. There was an observed synergy between the working groups, especially as the Universities Norway working group realized the need to extend the scope of their work beyond Open Science. The working group quickly realized that it is more efficient to evaluate openness as one parameter of each of the categories, rather than considering it as a category on its own.

The research assessment working group is composed of representatives from Norwegian universities, one research institute, The Young Academy of Norway, and The Research Council of Norway, the largest funder in the country. The working group acknowledged the difference between evaluating research projects for grant funding and evaluating academic staff in a university setting, so it formally established a university focus and perspective. While the government is not formally involved, they remain informed of the group’s progress.

The top-down methodology described here is relatively easy to adopt in Norway because of the small number of universities. However, the current goal of the working group is not necessarily to advocate for the amendment of any national laws or regulations. Instead, the working group is striving to change practice and build capacity in the long term by offering a practical assessment matrix as a framework for institutions to take in a broader set of evaluation criteria, as well as establishing concrete digital tools to easily access existing national databases, such as Cristin² and other national education and teaching databases.

Specific obstacles faced were: limited awareness of research assessment reform and its potential benefits; lack of evidence on potential benefits of research assessment reform; resistance to research assessment reform from academic leadership; resistance to research assessment reform from researchers; concerns over increased costs (e.g., skilled staff, support structures); complexity of research assessment reform (e.g., different national and disciplinary practices); lack of institutional capacity (e.g., skilled staff, support structures); and lack of coordination among the relevant actors within the institution.

There was a gradual change in Norwegian national laws regarding career assessment. Current government-directed practice is relatively holistic, often including a focus on narrative self-evaluation. The Universities Norway working group was formed in the fall of 2019. It anticipates the publication of a finished report by the end of 2020. They state, “it is expected that there will be various rounds of national consultations throughout 2021.”