Innovations in Responsible Research Assessment: Highlights from the DORA US Event

Enrique De La Cruz, former chair of the Molecular Biophysics and Biochemistry (MB&B) department at Yale University, detailed the department’s experimental use of an anonymized faculty hiring procedure. This effort was driven by the observation of persistent underrepresentation; in 2020, MB&B had 19 men (16 tenured) and 7 women (3 tenured), with only two members identifying as underrepresented minority (URM) groups out of 26 faculty members. De La Cruz noted he was the only URM hired in the department for its first 50 years.

The experimental process aimed to mitigate biases like the Halo Effect, Homophily, Availability Bias, and Racial/Ethnic Bias by requiring applicants to omit “powerful cues” such as their names, gender, race, the institutions they trained at, and the journals they published in.

Key procedural details changed included:

• Applicant Investment: Applicants were required to self-anonymize their materials, demanding an ‘investment’ in the process and forcing them to articulate their contributions without relying on professional ‘shorthand’ (e.g., publishing in a prestigious journal or working for a famous professor).
• Evaluation Structure: The search committee used rubrics to ensure consistency and equity. The first cut of candidates was made only using anonymized statements on past and future research, teaching, and community awareness, without reference letters or CVs. CVs and recommendation letters were only reviewed after candidates passed the anonymized screening.

Results of the 2020-2021 search showed a dramatic shift in the applicant pool:

• The percentage of self-identified URM applicants rose from 3 (less than 2%) in the previous search (2020) to 22 (11%).
• Women applicants increased from 31 (18%) in the prior search to 62 (32%).

The anonymization successfully enriched the diverse short list, resulting in women making up 47% (15 of 32 candidates) and URM members making up 22–25% (7-8 candidates) of the short list after the first cut. De La Cruz highlighted that the new procedure appeared to be effective, resulting in two offers, both extended to women, one of whom was a person of color.

Learnings for Future Searches:  De La Cruz noted that in subsequent searches, they realized two critical changes: scoring rubrics should be distributed to applicants so they know how they will be evaluated, and DEI/Community statements should not be anonymous (though separated for scoring) to avoid losing the profoundness and individual uniqueness of the applicant’s story.

De La Cruz also noted that while avoiding familiar shortcuts makes evaluation more work, this procedure helped remedy the self-inflicted notion that successful candidates from underrepresented groups might carry: that they were hired because of their identity rather than their merit.

Anna Hatch, Program Officer for Scientific Strategy at the Howard Hughes Medical Institute (HHMI), shared HHMI’s strategic, collective approach to responsible research assessment, recognizing that there is no panacea for deeply entrenched practices.

HHMI is shifting incentives through the following actions:

• Recognizing Preprints: HHMI has generally recognized preprints in researcher assessment since 2017.
• Prioritizing Preprint Versions: HHMI scientists subject to HHMI’s Immediate Access to Research policy are required to make articles that are submitted for publication to preprint servers or journals on or after January 1, 2026 and represent major contributions from their labs publicly available as both initial and revised preprints in accordance with this policy. For articles subject to this policy, HHMI’s general practice is to consider the most recent preprint version of the article as the official article version for internal reviews, not any subsequent journal publication.
• Deemphasizing Journal Names: Since 2024, HHMI has replaced journal names with article identifiers (PMIDs) in researcher bibliographies used for internal evaluations and competitions to encourage evaluators to focus on the science, not its place of publication.
• Transparent and Accountable Peer Review (TAP) Training:  This professional development program teaches graduate students and postdocs to write constructive, public peer-review reports on preprints. Focusing review on preprints helps early career researchers learn to evaluate the significance and importance of published research outside of the traditional journal mindset. The published reports receive a DOI and serve as scholarly outputs that can be added to researchers’ résumés or CVs.

Caitlin Schleicher (Open Research Community Accelerator, ORCA) introduced a critical funding mechanism designed to accelerate RRA implementation: the Modernizing Academic Appointment and Advancement (MA3) Challenge (https://www.ma3challenge.org/).

This initiative seeks “bold, creative strategies” to reform academic reward systems, specifically aiming to reimagine academic hiring, review, promotion, and tenure (RPT) processes. It addresses the known disconnect between what institutions value (like open science and public trust) and what they actually reward (like publishing in high impact factor journals).

The Challenge is organized by ORCA in partnership with the American Academy of Arts and Sciences and the Aspen Institute. It has secured $1.5 million collectively from the Gordon and Betty Moore Foundation, the Dana Foundation, the Rita Allen Foundation, and the Robert Wood Johnson Foundation.

The Challenge has four overarching goals, which applicants must meet:

1. To develop academic reward systems that foster a collaborative, responsive, and transparent research environment.
2. To tackle challenges in existing hiring and/or advancement language, processes, and policies.
3. To develop and foster capacity for institutional change work within the academy.
4. To advance beyond the discussion phase into implementation.

The Challenge offers two tiers of funding for U.S.-based accredited, tax-exempt colleges and universities:

• Tier 1: Up to $250,000 for colleges, universities, or larger divisions (e.g., schools, colleges, or cohorts of 3+ departments).
• Tier 2: Up to $50,000 for individual departments, centers, or institutes.

A core requirement is ensuring implementation-forward proposals, meaning no more than 50% of the award can be used for the planning phase (such as focus groups or surveys). Proposals must lead to explicit changes to RPT policies and practices. Partnerships, including those with professional societies, are encouraged.

Enrique M. De La Cruz, PhD, is the William R. Kenan Professor of Molecular Biophysics and Biochemistry and the Head of Branford College at Yale University. He is a first-generation Cuban-American who was raised in Newark, NJ. Dr.  De La Cruz received his undergraduate degree in Biology with a minor in Chemistry from Rutgers University, Newark College of Arts and Sciences, where he was inducted into Phi Beta Kappa and Beta Beta BetaHonor Societies. He earned his Ph.D. degree in Biochemistry, Cell & Molecular Biology (BCMB) at Johns Hopkins University School of Medicine and received postdoctoral training at the University of Pennsylvania School of Medicine. He joined the MB&B faculty as an Assistant Professor in 2001 and chaired the Department from 2000-2003. He was a Visiting Scientist at Centre National de la Recherche Scientifique (CNRS), Commissariat a l’Energie Atomique (CEA) & Universite Joseph Fourier in Grenoble, France, a Mayent-Rothschild Senior Researcher Fellow at the Institut Curie, Paris, and an Invited Professor Fellow at ESPCI Paris Tech (ecole superieure de physique et de chimie industrielles de la ville de Paris). In 2020, he was honored as Cell Press’ 100 inspiring Hispanic/Latinx scientists in America, in 2021 named an Inaugural Fellow of the American Society for Biochemistry and Molecular Biology (ASBMB) and a member of the Connecticut Academy of Science and Engineering (CASE), and in 2022 a Fellow of the American Association for the Advancement of Science (AAAS). Dr.  De La Cruz’s research focuses on the actin cytoskeleton, molecular motor proteins, and nucleotide signaling enzymes. Dr.  De La Cruz is actively involved with various scientific societies, journals, and peer review committees, and actively participates in outreach activities focused on enhancing minority participation, career development and retention in the sciences. 

Anna Hatch, PhD,  is a Program Officer for Scientific Strategy at the Howard Hughes Medical Institute (HHMI). In this role, she leads initiatives to improve research integrity and accelerate discovery through innovations in academic publishing and researcher assessment. In the past year, she has led the development of a peer review training program at HHMI, called Transparent and Accountable Peer Review, where graduate students and postdocs learn to write constructive, collegial public peer review reports on preprints. She previously served as the Program Director for the Declaration on Research Assessment (DORA), where she worked with the academic community to advance practical and robust approaches to researcher assessment, including the use of structured narratives. DORA engages the community to raise awareness of new tools and processes, facilitate the implementation of good practices, and catalyze change. Before working with DORA, Anna was a science policy fellow at Research!America, a nonprofit advocacy alliance to increase public and policymaker awareness of the benefits of medical research. She received her PhD in Biochemistry from Dartmouth College where her research focused on mitochondrial dynamics. In 2021, Anna was awarded the Rada Distinguished Alumni Award from the from the University of Wisconsin—LaCrosse recognizing her contributions to improving research culture.

Caitlin Schleicher, PhD, is the Director of Community & Partnerships at the Open Research Community Accelerator (ORCA). She previously served as Director of the Higher Education Leadership Initiative for Open Scholarship (HELIOS Open), where she led strategic planning and operational execution for over 100 college and university campuses working together to align open scholarship values and academic incentives. She has held positions at Johns Hopkins University in the medical library and in the president’s office as a Scholarly Communication Informationist. She holds a Bachelor of Arts in English and a Master’s in Library and Information Science from the University of Maryland – College Park.