Participant Commentary – Driving Institutional Change for Research Assessment Reform

In the fall of 2017, the Charité Universitätsmedizin Berlin introduced five additional items to the online application form for professorships. These new items are a) A narrative on the candidate’s overall scientific contributions, b) Statements on the impact of the candidate’s self-selected top 5 publications , c) A record of the candidate’s open science and reproducible research activities, d) Information on the candidate’s contribution to team science, e) Academic age. 

New faculty members at most (if not all!) research universities are given the same spiel: “your success here, and eventual promotion and tenure, is built on a three-legged stool, with one leg being research, one being teaching, and one being service.”  And shortly thereafter, well-meaning mentors and department heads will tell the newbie, sotto voce, “do as little service as you can get away with, preferably none, and the bar you have to meet for teaching is just, don’t suck!”

The precise path taken to implementing DORA will depend on the history and organisational idiosyncrasies of each institution. Nevertheless, it is likely that the establishment of an internal working group or committee to consider how best to infuse the spirit of the declaration within the institution will be a sensible move in most cases. This is the approach we took at my university, Imperial College London, after we signed DORA in January 2017.

When scientists publish a journal article, they are doing more than just disseminating their work: they’re attaching it to a journal title that will, rightly or wrongly, telegraph signals about its quality. Preprints help to unbundle communication from the other functions of journal publishing, and they allow evaluators—funders, hiring committees, and potential mentors—to read a candidate’s most recent work.

Unfortunately, science is rife with examples where research assessment diminishes diversity. Hiring, promotion, and grant decisions are made with incomplete information that is also poorly predictive of success—the perfect conditions for bias to emerge.

Scientific societies have a key role to play in changing and improving assessment of researchers. Many are key publishers of quality content and many of their journals are recognized as such without the burden of journal impact factors. They also play key roles in shaping the scientific culture of disciplines, including around ethics, authorship, and outreach, including in discussions at meetings and in career workshops.

There is widespread recognition that the research culture in academia requires reform. Hypercompetitive vying for grant funding, prestigious publications, and job opportunities foster a toxic environment. Furthermore, it distracts from the core value of the scientific community, which is a principled search for increasingly accurate explanations of how the world works.

Having committed to the hiring and development of early career scientists, it is in the best interest of departments and institutions to make the tenure process as transparent and consistent as possible to ensure success. One mechanism to accomplish this is to allow untenured faculty to discuss and vote on the tenure files of more senior faculty members.

Faculty often cite concerns about promotion and tenure evaluations as important factors limiting their adoption of open access, open data, and other open scholarship practices. We began the review, promotion, and tenure (RPT) project in 2016 in an effort to better understand how faculty are being evaluated and where there might be opportunities for reform. We collected over 800 documents governing RPT processes from a representative sample of 129 universities in the U.S. and Canada.

Universities cannot achieve their missions and visions if their stated values are out of line with research assessment policies and practices. Although most university mission statements specify research, teaching, and public service as their central commitments, contributions to research are often valued at the expense of teaching and public service. How serious is this misalignment and what can be done about it?