At the end of each year, the DORA steering committee and advisory board reflect on progress to improve research and researcher assessment. Even though 2020 presented no shortage of challenges stemming from the global COVID19 pandemic, organizations made time to experiment with policies and practices, develop guidance, and create new tools to support the development of fair and responsible research assessment.
The following list of new developments and essential reading was created with input from the steering committee and advisory board. While the search was extensive, it was not comprehensive. Please let us know if there are other advances that we should add to our collection of community resources.
From DORA:
- In March, DORA launched a quarterly virtual meeting for public and private funders of research in the Americas, Europe, and Africa to discuss new practices and increase communication for research assessment reform. DORA organized its first complementary meeting in August for research funders in the Asia-Pacific region.
- DORA published a framework for driving institutional change for academic assessment in August based on discussions at a meeting convened by DORA and the Howard Hughes Medical Institute (HHMI) in 2019. The framework provides four goals for institutions seeking to improve academic assessment: 1) understand obstacles; 2) experiment with different ideas and approaches; 3) create a shared vision; and 4) communicate that vision on campus and externally to other research institutions.
- As another outcome of the meeting DORA co-sponsored with HHMI, DORA began collaborating with Ruth Schmidt, Associate Professor at the Institute of Design of the Illinois Institute of Technology to develop a toolkit of resources to help academic institutions improve their policies and practices. So far, we have created two briefs together:
- Rethinking Research Assessment: Ideas for Action outlines five common myths about research evaluation to make the case for reform and offers five design principles to help institutions experiment with and develop better research assessment practice.
- Unintended Cognitive and Systems Biases identifies four infrastructural implications of common cognitive biases and provides strategies to develop new institutional conditions that reduce bias.
- In collaboration with the European University Association and SPARC Europe, DORA developed a digital repository of 10 case studies that examine key elements of institutional change for academic assessment reform, including motivations, the people involved, processes, and timelines for achieving specific goals. The repository is intended to be a useful tool for the scholarly community by providing clarity on how and in which context assessment reform is being developed and identifying potential pathways for change.
From elsewhere:
- The Global Research Council organized a virtual conference on responsible research assessment in November to explore the research ecosystem as a whole, the stakeholders who influence and impact research assessment, and the roles funders play in assessment. In advance of the meeting, the Research on Research Institute in partnership with the Global Research Council, DORA, UK Research and Innovation (UKRI), and the National Research Foundation in South Africa developed a white paper titled “The changing role of funders in responsible research assessment:progress, obstacles and the way ahead.”
- In June, Wellcome published guidance for research organisations on how to implement responsible and fair approaches for research assessment. Starting in January 2021, Wellcome-funded organizations will be asked to publicly commit to assessing research outputs and other research contributions based on their intrinsic merit and discouraging the inappropriate use of proxies or metrics.
- The Latin American Forum for Research Assessment (FOLEC) from the Latin American Council of Social Sciences (CLACSO) produced a series of documents and held regional and international meetings to discuss and advance the transformationation of research assessment in Latin America and the Caribbean.
- The Yale Molecular Biophysics and Biochemistry department announced a radical new approach to their faculty search in October: “This year, candidates for tenure-track Assistant Professor positions in MB&B will be asked to submit anonymized applications—no names of people, places, funding agencies or journals.”
- The disproportionate impact of COVID-19 on women and early career researchers was identified as an area where the pandemic may create downstream consequences for inequity in research and researcher assessment.
- The health disparities highlighted by the COVID-19 pandemic and demonstrations for racial justice called attention to longstanding systemic racial inequities. As part of a call to action for structural change within academia, more than 9,000 people used the #BlackInTheIvory hashtag on Twitter.
- Science Europe issued a new position statement and recommendations on research assessment processes in July. The statement and accompanying recommendations seek to promote assessment of research quality that is effective, efficient, fair, and transparent.
- A new Chinese national policy issued by the Ministry of Education and the Ministry of Science and Technology in February directs institutions not to use the number of papers a researcher has published or their citations as the sole criteria for recruitment or promotion.
- The Hong Kong Principles for assessing researchers, which include valuing complete reporting, rewarding the practice of open science, acknowledging a broad range of research activities, and recognizing essential other tasks like peer review and mentoring have seen initial endorsement by 15 institutions in 2020.
- Research funders一The Dutch Research Council, Science Foundation Ireland, Swiss National Science Foundation, and UKRI一are adopting the use of narrative CV formats to balance quantitative and qualitative assessments.
Essential reading
Books
- In Transforming Research Excellence: New Ideas from the Global South
By Kraemer-Mbula, Erika, Tijssen, Robert, Wallace, Matthew L., & McLean, Robert. - Reassembling Scholarly Communications: Histories, Infrastructures, and Global Politics of Open Access
Edited by Martin Paul Eve and Jonathan Gray
Articles
- Research integrity: nine ways to move from talk to walk
By Niels Mejlgaard, Lex M. Bouter, George Gaskell, et, al. - University rankings need a rethink
By Elizabeth Gadd - The transformative power of values-enacted scholarship
By Nicky Agate, Rebecca Kennison, Stacy Konkiel, Christopher P. Long, Jason Rhody, Simone Sacchi & Penelope Weber - Seven intersectional feminist principles for equitable and actionable COVID-19 data
By Catherine D’Ignazio and Lauren F. Klein