This is part of DORA’s toolkit of resources to support academic institutions that are improving their policies and practices. Find the other resources in the toolkit here.
Balanced, broad, responsible: A practical guide for research evaluators is a short, informative video that is accompanied by a one-page brief. The video and document are meant to serve as resources for public and private funders of research seeking to promote a more holistic approach to the evaluation of funding proposals.
These resources provide a brief explanation of why there is a need to shift how funders evaluate research:
- Research assessment is central to research culture.
- An overemphasis on quantitative criteria hinders the ability of the evaluator to holistically assess an applicant.
- Like research itself, the way research is assessed needs to evolve via experimentation and the introduction of new ideas.
These resources also provide a ‘checklist’ of six practical suggestions for research funders who are seeking to implement or improve responsible assessment of funding applications:
- Align decision-making to strategic objectives and specific criteria of the funding institution or instrument.
- Be clear about the context and limitations of any quantitative metrics used and balance them with qualitative parts of the funding proposal.
- Look broadly to include research activities beyond publications and grants to capture the full range of a researcher’s contributions.
- Be aware of biases that arise from scientific and cultural stereotypes.
- Promote personal and group accountability for responsible research assessment during evaluation.
- If you are not sure whether you have a conflict of interest or not, ask the funding institution for guidance.
The one-page brief includes suggestions for use within multiple contexts, including sharing the video and brief with grant evaluators, institutions, and researchers.
These resources are a result of DORA’s partnership with the Luxembourg National Research Fund (FNR). The checklist was developed and iteratively improved with the valuable feedback from members of the DORA Steering Committee, DORA funder Discussion Group, active researchers, and other funding agencies.