By Olivia S. Rissland (University of Colorado School of Medicine)
Unfortunately, science is rife with examples where research assessment diminishes diversity. Hiring, promotion, and grant decisions are made with incomplete information that is also poorly predictive of success—the perfect conditions for bias to emerge. Metrics will naturally be weighted differently for different individuals, but what can be most telling are those individuals who are given a pass for specific metrics (e.g., they are hired for their potential) and those who are not (e.g., they are required to pass a threshold or show a track record). Combined with natural homophily and the make-up of committees making these decisions, bias then becomes a powerful force in opposition to diversity. To put it another way, often search committees don’t hire the person who fits the job, but rather they make the job fit the person.
Institutions and funding agencies can take immediate actions to mitigate the impact of bias. These include (but are not limited to):
- Remove institutional nominations for awards. University gatekeepers can bias the pool of applicants that the selection committee sees. Because institutions are often restricted to one or two nominees, it is easy for bias and unfair procedures to lurk in the small sample size.
- Track and publish the make-up of the applicant and interview pools as well as hires/awardees. Transparency about the fairness of selection procedures is critical not just for accountability, but also for garnering trust with underrepresented communities. For instance, awards that show an overwhelming male bias can lead to an underrepresentation in women applicants because applicants believe (possibly erroneously) that the selection committee is biased; this response, of course, further compounds the issue.
- Ensure that interview pools are diverse. The interview stage thus represents the critical step to allow women and minorities to have a fair shot at being hired and selected. A recent study has shown that when only one interviewee was a woman, they had <5% chance of being hired. In contrast, when two interviewees were women, there was a 50% that a woman would be hired [1].
- Make award eligibility criteria less restrictive. Age limits, time from Ph.D. limits, and nationality requirements all restrict the pool of people who can apply for awards, and these effects can then be accentuated by the Matthew effect (i.e., where the rich get richer). For instance, the NIH ESI eligibility window is 10 years from completion of the terminal degree, but an alternative criterion is that used by CIHR (5 years from starting a tenure-track position), which does not negatively affect faculty who had long postdoctoral training.
More broadly, any policy change in research assessment must be considered from the perspective of diversity. While the impact of any individual job searches or grant decision can be relatively minor, the Matthew effect can further entrench the effect of bias in research assessment. A critical step is to talk to many different scientists during policy development and take their concerns seriously, especially when specific groups highlight potential pitfalls or unintended consequences. Many policies have the potential to be weaponized against the people who have been historically kept out of science, especially if there is poor implementation, and so all policy decisions need to be explicitly considered for their impact on diversity.
[1] https://hbr.org/2016/04/if-theres-only-one-woman-in-your-candidate-pool-theres-statistically-no-chance-shell-be-hired