DORA launched a new virtual discussion series for public and private research funders on Wednesday, March 26. The goal of the series is to increase communication about research assessment reform by providing a space for funders to share and discuss new initiatives. We hope this will ultimately serve as a platform to accelerate the spread of good research assessment policies and practices.
Representatives from the Swiss National Science Foundation, Dutch Research Council, and Wellcome Trust provided updates on some of their pilot projects:
Swiss National Science Foundation
The Swiss National Science Foundation is piloting a new online CV format called “SciCV” to increase consistency in decision-making. Deputy Head of Strategy Support Michael Hill acknowledges that individual track records are diverse, but notes that CVs shouldn’t be, so that reviewers can better compare candidates. Members of the Swiss National Research Council played an important role on the task force responsible for developing the SciCV. Hill believes that this community involvement led to greater acceptance of the new format, because researchers felt their opinions were represented.
SciCV integrates with ORCID, making it easy to populate if you have an ORCID ID. Because SciCV is a pilot project, Hill explained that they are working with researchers at Leiden University to evaluate its effectiveness as a tool for responsible research assessment by interviewing peer reviewers and reviewing the procedures and processes related to its use.
The Dutch Research Council
The Dutch Research Council (NWO) is part of a national effort to rethink research assessment, primarily by expanding the types of knowledge and expertise considered for hiring, promotion, and funding decisions. The position paper, Room for everyone’s talent, published in November 2019, was the result of a collaboration between public knowledge institutions and research funders in the Netherlands. Policy Officer for Open Science and Quality Hans de Jonge explained the position paper was a starting point to establish common goals and an agenda. It identifies next steps for each organization involved and communicates them to the Dutch academic community.
For their part, NWO used a national conference to solicit feedback on their review processes. This resulted in removing references to unhelpful metrics in application forms, instructing referees to change focus, and introducing a narrative CV structure. Policy Officer of the Talent Scheme Kasper Gossink-Melenhorst said that NWO recognized there was no “ideal type” of academic. The structured narrative format provides more room for researchers to show what they do is exceptional. Some worried that giving up metrics increases the time spent evaluating candidates, but Gossink-Melenhorst says that time is made up for by reducing the length of the CV to 2-3 pages and capping the number of key outputs at 10.
The new CV format was evaluated at NWO and in collaboration with a researcher at the University of Amsterdam, who found no major negative effects and concluded that the CV led to an increased consensus between reviewers. NWO notes that the new CV format leads to a more diverse selection of types of researchers. Most panel members reported an appreciation for a more holistic approach, but there is concern that many researchers focus on the “rules” rather than the intent behind them.
Wellcome Trust
The new Open Access Policy at the Wellcome Trust will come into effect January 2021. As part of the policy, Program Manager David Carr said that Wellcome is requiring all of its funded universities and research institutions to evaluate researchers based on the intrinsic merit of their work, rather than on the title of the journal or the publisher. Wellcome is in the process of developing guidance to support these organizations in making the transition (see: here).
The guidance provides three overarching steps:
- Statement of support
- Implementation plan and timeline
- Plan to assess implementation
Wellcome asked the academic community for feedback on the guidance and received over 50 responses. They are in the process of reviewing the responses, and Carr hopes the final version will be published by the end of April.
DORA’s funder discussion group is a community of practice that meets virtually every quarter to discuss policies and topics related to fair and responsible research assessment. If you are a public or private funder of research interested in joining the group, please reach out to DORA’s Program Director, Anna Hatch (info@dora.org). Organizations do not have to be a signatory of DORA to participate.