Responsible Research Network, Finland

The Good Practice in Researcher Evaluation: Recommendation for the Responsible Evaluation of a Researcher in Finland are guidelines to improve how researchers are assessed in Finland. The Recommendation provides a set of general principles (transparency, integrity, fairness, competence, and diversity) that apply throughout 13 recommended good practices to improve four aspects of researcher evaluation, including:

• Building the evaluation process
• Evaluation of research
• Diversity of activities
• Researcher’s role in the evaluation process

The aim of the Recommendation is to encourage all those involved in the researcher evaluation process to consider how the process impacts the development of the research community as a whole, and how to achieve the desired outcome in the most responsible way.

The Recommendation pays particular attention to the culture change required in evaluation as a result of new ways in which research is conducted and communicated. For example, new forms of research knowledge sharing, changes in research processes, multidisciplinary and new phenomenon-based research methods, as well as digitalization, have all had an impact on researchers’ work. Because of this, the evaluation culture needs to align with the developing research culture.

Research transparency, ethics, diversity, and societal interactions are important themes throughout the Recommendation. In addition, the role of the researcher in the larger community, and as a teacher and mentor, has been given a central role in the Recommendation.

In 2020, the Declaration for Open Science and Research (AVOTT, 2020), was collaboratively drafted by the National Open Science and Research Steering Group organizations, that included the Federation of Finnish Learned Societies, Universities Finland UNIFI, and many others. Currently, more than 70 Finnish organizations have formally committed to the vision, strategy, and policies of the Declaration approved by the AVOTT steering group. One key strategic goal of this Declaration includes building responsible policies for open scholarship. In this light, in 2022, The Policy for Open Scholarship (AVOTT, 2022) was published with the objective of responsible assessment that “the organization has at its disposal practices, criteria and a knowledge base for documenting diverse outputs and merits that promote open science and its culture as part of the assessment and merit of Finnish research organizations and their personnel”. To achieve this objective by 2025, the policy also outlines seven required actions, including “ensure evaluation of research and a researcher in accordance with the national recommendation”, and the appointment of a “responsible party that researchers can contact should there be shortcomings in the responsibility of the evaluation”.

A major driver for the national Recommendation was the need to make deliberate and conscious decisions regarding goals, criteria, data, and methods in evaluation processes. The Recommendation represents an attempt to “change and make more responsible” evaluation practices compared with the ones currently in place.

Additionally, there was a desire to foster a research culture embodying the desired principles of transparency, integrity, fairness, competence, and diversity. The Recommendation focuses on providing guidance to reflect on and implement their responsible evaluation practices.

One success was the broad and enthusiastic adoption of the principles behind the Recommendation. For example, the Academy of Finland, the main national funder for basic research, confirmed support for responsible evaluation of researchers. An initial challenge was to gain a consensus of opinion across all fields and a broad range of stakeholders, which was ultimately addressed through public consultation and discussion.

As for The Finnish Declaration for Open Science and Research, it represents the Finnish research community's response to the international open science policies that bind Finland as an EU Member State and its research community internationally.

The Federation of Finnish Learned Societies formally initiated a national task force to consider responsible research evaluation.

A national task force was founded based on shared concerns identified by learned societies, research funders, policy organizations, publishers, national open science coordination, and the national research integrity board. While many national entities were involved in the Recommendation’s creation, the approach is considered “bottom-up”; in Finland there is a historic culture of autonomy for academic stakeholders.

The Responsible Researcher Evaluation task force facilitated necessary dialogue with researchers across academic disciplines in order to obtain genuine buy-in. The entire Finnish research community, including HR staff, librarians, and researchers, were invited to comment on an initial draft of recommendations. The draft recommendation was opened to public consultation, and over 50 research-related organizations participated. The task force developed the Recommendation with the feedback from the research community.

In addition, the Recommendation timing coincided with the uptake of FAIR (findable, accessible, interoperable, and reusable) data and open science initiatives in Finland. These initiatives incentivize and reward researchers for producing open and FAIR data, and align with the Recommendation. In the coming years, the focus will be on building the capacity to move evaluation practices beyond quantitative publication metrics and in closer alignment with the goals of the Recommendation.

The Federation of Finnish Learned Societies was fortunate in that it is a trusted and independent entity, and it was within their purview to initiate the task force and propose the Recommendation. Additionally, they had the respected authority necessary to build a broad coalition of stakeholder groups to develop the Recommendation.

Specific obstacles faced were: lack of evidence on potential benefits of research assessment reform; resistance to research assessment reform from researchers; concerns over increased costs; complexity of research assessment reform; lack of coordination among the relevant actors within the institution; and lack of reliable data sources for assessment.

In addition, these efforts needed to be backed up by a constant monitoring model to successfully realize the implementation of Open Scholarship and Open Science objectives in organizations. Thus in 2022, the Open Science coordination also launched a biannual National Open Science Monitoring (AVOTT, 2022). The aim is to support research organizations in Open Science activities, strategic negotiations with ministries, monitoring reaching national open science objectives, as well as providing information for international comparisons. The monitoring is largely based on a questionnaire to organizations, and it includes questions on responsible assessment, such as does the organization have internal guidelines to support the implementation of the Recommendation for the responsible evaluation of researcher in Finland and has the organization appointed the responsible party that researchers can contact in case of misconduct? This questionnaire is based on principles and themes also overlapping with that in The Self-evaluation tool for culture of open scholarship services, which was created with the intent to help organizations self-assess themselves and their development of services, while the monitoring model makes it possible to assess the maturity level of the entire open science field.

The monitoring exercise is not designed to provide an indicator of open science to be used in the performance-based funding model, which the Ministry of Education and Culture uses to allocate three-fourths of core-funding annually to the universities (Ministry of Education and Culture, 2018). However, annual core-funding is partly based on strategic development agreements between the ministry and the universities, and the “research organizations have the possibility to choose open science and research indicators in their bilateral negotiations with the ministry” (AVOTT, 2022). The Finnish case shows an example of how self-regulatory policies and mechanisms can be created and used by the research community itself for promoting and monitoring RRA, and potentially recognized by the ministry as the basis of institutional funding negotiations. The results of the Monitoring of open science and research can be found here.

The formation of the task force by the Federation of Finnish Learned Societies in 2018 was preceded by a long period of informal contact and working through existing networks. Once the task force was created, the policy development process took 12 months. The working group published their recommendations in a report in early 2020.

In 2020, the task force completed its work, and a steering committee was subsequently formed to implement the Recommendation and develop impact measures. The Finnish national consortia are working on developing the data architecture and models necessary to encourage the Finnish research and funding organizations to change and improve their evaluation practices. In December 2020, Universities Finland endorsed the Recommendation in their theses on sustainable development and responsibility.

The Finnish Responsible Research Network is building policies, guidelines and tools to robustly integrate and monitor Openness in their research culture aligning with the missions stated in the Declaration by 2025.