The COVID-19 pandemic has influenced many facets of the research landscape, including the manner in which researchers disseminate and gather feedback on their findings. Increasingly, researchers have been sharing preprints for the rapid dissemination of information to the public and policymakers. Recent surges in the use of preprints, spurred by the COVID-19 pandemic, have highlighted the need to consider how these research outputs should be recognized and rewarded in academic promotion, tenure, and hiring practices. In particular, the recognition of preprints offers value to early career researchers, whose career progression and ability to obtain funding may be negatively impacted by the extended review time and multiple rejections typical of journal publications.
The prioritization of publication quantity and journal metrics in traditional academic assessment practices do not tend to reward outputs and practices (e.g., data sharing, preprints, reproducibility) that have been valuable during the pandemic. So on June 29, 2021, DORA and ASAPbio held a joint webinar to equip early career researchers, faculty, and academic leadership with knowledge to advocate for the use and evaluation of preprints as part of academic assessment practices at their institutions.
In this webinar, a group of experts presented on the benefits and the potential for incorporating preprints into academic hiring and promotion criteria. Speakers included Needhi Bhalla, University of California, Santa Cruz, United States; Denis Bourguet and Thomas Guillemaud, Peer Community In, France; and Sarvenaz Sarabipour, Johns Hopkins University, United States. The webinar was moderated by Haley Hazlett, DORA Program Manager, and Jessica Polka, ASAPbio Executive Director. The speakers discussed the value of considering preprints in academic assessment for minoritized groups and early career researchers. They also presented on burgeoning methods for evaluating preprint quality.
Fostering trust in preprints
Preprints offer a range of benefits, serving as a means for rapid and free communication of research, and allowing researchers to establish priority for scientific discoveries and evidence of productivity. Although preprints may be a force for good, Guillemaud discussed the importance of creating spaces for preprint evaluation that preserve the benefits of prompt dissemination while also maintaining preprint quality and providing authors with feedback. Guillemaud introduced the “Publish-Review-Curate” model proposed by Stern and O’Shea in 2019. Within this model, author preprints are made public, can be reviewed by experts through a particular service such as eLife or Peer Community In (PCI), and given an endorsement by the service based on the preprint reviews. For example, PCI is a free service that curates communities of editors (“recommenders” in the PCI vocabulary) who provide recommendations for whether the preprint should be endorsed by the service, based on evaluation by peer reviewers. Importantly, the endorsement process is not necessary for the preprint to be posted. Finally, the use of recommender endorsements can help foster trust in preprints, which is an important factor to consider when advocating for preprint incorporation into academic assessment practices. An example of this in action is that a number of French universities now accept preprints recommended by PCI as part of their doctoral graduation requirements.
Preprints for diversity, equity, and inclusion
Bhalla addressed the importance of reforming assessment practices as a means to improve equity in faculty hiring. As outlined in Bhalla’s 2019 perspective piece, Strategies to improve equity in faculty hiring, there remains a persistent gap in the number of biomedical faculty from underrepresented or minoritized groups: “the lack of diversity among biomedical faculty is not because there are not enough qualified trainees from underrepresented groups, which is often referred to as a “pipeline” issue but rather that our system of faculty hiring, retention, and promotion maintains homogeneity at the expense of increasing diversity and improving equity.”
When confronted with these challenges to improve equity, Bhalla pointed out that the indicators of excellence and merit need to be redefined as they are inherently flawed. Here, Bhalla identified preprints as a powerful resource for minoritized early career researchers, giving them the ability to enter the faculty job market with tangible evidence of their contributions without the sometimes years-long waiting periods that come with traditional journal publication. Additionally, the use of preprints can help to remove unintended biases that are typically coupled with journal metrics and journal name-recognition. This would allow research to be assessed on its own merit, rather than flawed proxies of quality. Importantly, assessing preprints for promotion and hiring still requires equity efforts since information such as geographical location or affiliation may still result in unconscious biases during assessment.
Preprints for early career researchers
Sarabipour rounded out the discussion with a presentation on her work advocating for the utility of preprints for early career researchers. As outlined in Sarabipour’s 2019 perspective, scientific outputs are varied and range from research articles to patents to software development and beyond. Despite the wide range of outputs that researchers are expected to generate, assessment frameworks are often limited to their record of peer-reviewed publications. Reiterating Bhalla’s point, Sarabipour highlighted the limitations of conducting assessments using such biased criteria. In particular, these criteria introduce challenges to early career researchers who rely on the dissemination of their work to gain feedback and recognition. Preprint servers can help early career researchers by allowing them to make their work immediately available.
Sarabipour also discussed the results of a survey conducted among participants of the Future PI Slack group, a community of early career researchers seeking faculty positions. The survey found that, of the 317 responses, 55% of respondents had posted at least one preprint, and general feedback from the community showed that early career researchers value preprints as tangible proof of their work. Interestingly, 10 of 15 of the faculty search committee members surveyed confirmed that preprints were generally viewed favorably by search committees, although published papers and journal metrics still carried more weight. Sarabipour pointed out, however, that a multitude of factors that play a role in hiring and promotion decisions need further investigation and consideration. Additionally, Sarabipour suggested that normalizing preprints may help increase the number of women in faculty positions, as touched on earlier in the webinar by Bhalla. In conclusion, Sarabipour suggested that normalizing the use of preprints may be achieved via the Plan U approach, which mandates that funders require grantees to post their manuscripts first on preprint servers. This may help address faculty hesitancy to allow their students and postdocs to use preprints, and increase acceptance of preprints as part of hiring and promotion practices.
Haley Hazlett is DORA’s Program Manager.