DORA’s ultimate aim is not to accumulate signatures but to promote real change in research assessment. One of the keys to this is the development of robust and time-efficient ways of evaluating research and researchers that do not rely on journal impact factors. We are keen to gather and share existing examples of good practice in research assessment, including approaches to funding and fellowships, hiring and promotion, and awarding prizes, that emphasize research itself and not where it is published.
If you know of exemplary research assessment methods that could provide inspiration and ideas for research institutes, funders, journals, professional societies, or researchers, please contact DORA.
University of California, Berkeley
Department of Molecular and Cell Biology & Helen Wills Neuroscience Institute
Applications for assistant professor positions were designed to highlight the significance of an applicant’s accomplishments rather than default to using journal-based metrics as a substitute for research quality. The advertisement asked applicants to summarize their major research accomplishments, ongoing and planned research program, and contributions to diversity. Applicants were also asked to select three significant articles from their list of publications and describe the impact of each.
University College London
University College London (UCL) released its Academic Careers Framework, which provides information related to its promotion processes. In it, the framework recognizes that UCL is a signatory of DORA on page five and as such states that it rejects the use journal-based metrics to quantify the quality of research in question.
University of Colorado School of Medicine
The Faculty Promotions Committee at the School of Medicine advises candidates against using journal-based metrics in their promotion or tenure dossiers, as these metrics do not accurately capture the significance of specific research contributions. Instead, the emphasis is on writing a narrative statement that clearly conveys the significance of the work. While metrics like the H index or total citation counts can be used to help evaluate the impact, they cannot serve as a replacement for a clear description of the work’s value. In addition, the committee recognizes value from all outputs and outcomes generated by research.
My career focuses on scientific research. How should I document my research accomplishments in my promotion or tenure dossier?
The Faculty Promotions Committee discourages the use of journal-based metrics (i.e., journal impact factors), since it is the quality and importance of the research contribution itself that is the key. Research importance can also be measured by its impact on policy, practice or the scientific discipline. Other outputs from scientific research, such as intellectual property, databases, software or others, may also be highlighted.
University of Glasgow
In October 2017, the policy for the Use of Quantitative Indicators in the Assessment for good research practice was created to apply fair and transparent mechanisms to measure researcher performance in hiring and promotion at the university. The policy recognizes high-quality research is important to the mission of the university and says article-level metrics are more appropriate than journal-based metrics in research assessment. It explicitly states that Journal Impact Factor will not be used as the sole metric to evaluate research. The section providing context for the guidelines specifically names the San Francisco Declaration on Research Assessment.
University Medical Center Utrecht
In order to create a culture around research assessment that is free from metrics including Journal Impact Factor, the university held a series of meetings to facilitate discussions with researchers on the best ways to create change. Representatives from multiple career-stages were invited to discuss policies to define and measure societal impact as well as research excellence. Central to the success of the policy was the inclusion of researchers and faculty in these conversations, because their approval signified an agreement to be judged by the criteria. The university now favors the use of bio-sketches, where scientists summarize the impacts of their contributions.
UT Southwestern Medical Center
Department of Cell Biology
The cover letter plays a significant role in the initial assessment of candidates for assistant faculty positions in the Department of Cell Biology at UT Southwestern Medical Center and is used to reduce some of the biases associated with CVs, such as research pedigree or whether candidates have published in brand name journals. In the cover letter, applicants describe the significance of their work and what they envision for their research program. The department then uses short Skype interviews as an intermediate stage and screens as many as 30 individuals. Applicants are asked to prepare answers to two key questions for the Skype interview: 1) Where will your research program be in five years? and 2) How can UT Southwestern help you get there? Having the questions beforehand allows the department to identify more thoughtful candidates in addition to those who can think quickly.