By Miriam Kip and Ulrich Dirnagl (Berlin Institute of Health)
In the fall of 2017, the Charité Universitätsmedizin Berlin introduced five additional items to the online application form for professorships. These new items are a) A narrative on the candidate’s overall scientific contributions, b) Statements on the impact of the candidate’s self-selected top 5 publications , c) A record of the candidate’s open science and reproducible research activities, d) Information on the candidate’s contribution to team science, e) Academic age.
What sparked the change in the application criteria?
An increasing number of publishers and funding agencies are taking steps to encourage robustness and transparency in order to increase the probability that research results will be reproducible and have an impact on medical care. Academic institutions need to play their part in this international drive to increase the value and reduce waste in research. To this end, the QUEST Center for Transforming Biomedical Research was founded by the Berlin Institute of Health to improve the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative. Among other activities, this provided the basis for a reform of institutional research assessment processes at its University hospital and medical school, Charité Universitätsmedizin Berlin.
MERIT-PROF is an iterative policy implementation project led by the QUEST Center. The project aims to improve the assessment of research by implementing and evaluating the uptake of the new items into the hiring process for professors.
Facilitating patient-oriented translational research is a core mission of BIH and Charité. It is the task of the institution to provide the framework in which robust and reproducible research is incentivized and rewarded. We aim at transitioning from assessments of productivity and reputation based on metrics and narrow expert opinions to more transparent and structured assessments of the research robustness and reproducibility based on the content.
The organizational context
The QUEST center is one of the two innovation drivers at the BIH. The BIH is a public institution that obtains 90% of its funding from the federal government. The Charité Universitätsmedizin Berlin is a large academic medical center that employs more than 4,000 researchers and physicians and teaches 7,500 students. The Charité is one of BIH’s two corporate institutions (along with the Max Delbrück Center for Molecular Medicine (MDC)). Each year Charité hires 25 to 35 professors through external calls. The number of applications varies, from one to over 50 applications per call. The BIH and Charité share a common office responsible for the management and organization of all calls and the hiring commission for professorships under the leadership of the faculty management (recruitment office). The MERIT project team serves as a consulting party to both and one QUEST member is participating in hiring commissions and during deliberations as an independent party without a vote.
Where are we are at now and what are the next steps?
By adding the new items, the whole dynamic of the process is supposed to change for the hiring commission but also for the operational teams (e.g. the recruitment office) who actually provide the documents and information on which the assessment through the hiring commission is based. While having mainly focused on and compared numbers usually displayed in graphs so far, now the hiring commissions are asked to evaluate the additional text information for up to 50 applicants as well. To facilitate the access to the new items, the MERIT project team together with the BIH/Charité hiring office currently develop a format that brings together the new items within the overall narrative as the starting point of the evaluation process and selects metrics (such as the Relative Citation Ratio) as auxiliary information.
The new format produces new workflows. These new workflows need to be integrated into the existing highly formalized procedures.
During the pilot phase, one QUEST member sat in selected hiring commission at different stages of the selection process mainly to observe and gain an understanding of the institutional procedures and common practices. During interview rounds, the QUEST representative for example suggests to the hiring commissions to include the Opens Science track record of candidates as one additional item to the selection procedure.
MERIT follows a bottom-up, participatory approach with the target groups (e.g. recruitment office, faculty) being the main actors implementing the policy. The stages of the implementation are need and field assessment, agenda and strategy setting, introduction to the field, and adjustment. The participatory approach provides a framework for trust building and acceptance of the new measures, and ensures the responsiveness of the policy to the actual needs, requirements and capacities in the field. Both are essential pillars for an effective uptake of the new approach and ultimately research assessment reform.
MERIT-PROF is a subproject of BIH MERIT and one of the four projects funded through the Wellcome Trust Translational Partnership with the Charité. The funding period will start in December 2019. In the next two years, MERIT aims at participating as (independent) advisors in at least 60% of all hiring commission. In this process, the assessments formats will be further developed and the uptake of the new items and their possible effect will be evaluated (process and outcome evaluation).