The November 2024 meetings of the Asia-Pacific and the Africa Americas Europe Funder Discussion Groups featured a presentation from the Global Research Council (GRC) working group on responsible research asssessment (RRA) Presenters were Claire Fraser (UKRI), Joanne Looyen (MBIE) and Anh-Khoi Trinh (NSERC).
GRC working group on RRA
GRC intends for their participants to adopt research assessment practices that promote and celebrate the varied aspects of research excellence, contributing to a more supportive and vibrant research culture that prioritizes high standards and rigorous, high-quality research. GRC is well positioned to be a leading advocate for responsible research assessment (RRA) with membership from 19 countries, 20 funding organisations and representing every region of GRC participants. An initial survey on RRA was undertaken in 2020, presented at the GRC RRA conference of the same year, and resulted in the launch of the working group. In 2024 the working group released an Action Plan, a report on the Dimensions of RRA, launched a framework for case studies on RRA, and undertook a second RRA survey with the Research on Research Institute (RoRI).
Dimensions of RRA
The first objective of the working group, “to advocate globally for the importance of RRA for improving the quality of research assessments” and to “work towards a shared understanding and goal for RRA…” is manifest in the document Dimensions of Responsible Research Assessment. Compiled from diverse geographic representation, and as such applicable and helpful for funders worldwide, the report highlighted 11 dimensions of RRA. The dimensions were categorized into three high-level categories: guiding principles (including integrity and responsible conduct; incentives to practice open research; equity diversity and inclusion; responses to the effects on research of global emergencies): governance and strategy (administration and monitoring; research assessment reform; influence on policy and practice): and process and methodology (research assessment criteria; evaluation of research contributions; impact assessment; reviewer and panel recruitment and training.) The Dimensions are a guide for funders to develop their own research practices. They are not intended to be prescriptive.
Funders have a unique and important role in promoting policies and frameworks that support research integrity and encourage open research, equity, diversity, and inclusion. Opportunities for all types of researchers need to be prioritised and barriers removed.
Case Studies
The second objective of the working group is to provide guidance and support to member organisations on how to embed RRA in their practice. To this end the working group has developed a template for case study submissions and is gathering examples from its members. Planning, implementation, monitoring and evaluating strategies are being compiled. Case studies will be collected from October 2024 to January 2025 representing a wide variety of funders and showing examples of each of the dimensions of RRA. A case study booklet will be presented at the 2025 Annual Meeting, and individual case studies will be presented via webinars etc.
The submission template for case studies can be found at https://globalresearchcouncil.org/about/responsible-research-assessment-working-group/
Research on Research Institute (RoRI)
Founded in 2019 at the Wellcome Trust, RoRI is now hosted by University College London. Supported by 19 organisations, many of whom are members of GRC, there are currently 42 researchers working across 10 research projects. RoRI projects look at research systems and cultures, gathering evidence to improve how research is funded, practiced, disseminated and evaluated.
The original survey for the GRC RRA working group was sent out in 2020. A second survey has been circulated this year – GRC participants are welcome to participate (closing date 6 Jan 2025), please email GRC-RRA@ukri.org. Results from the two surveys will allow for comparative analysis and results will be presented at the Annual Meeting. The 2024 survey is also collecting information on AI use in research funding.
Next steps for the working group
- Assess the results of the 2024 survey
- Extend the activities of the working group beyond the original 2025 deadline
- Collect more case studies and produce booklet for the Annual Meeting to be held in May 2025
- Develop a self-assessment tool and roadmap for funders for assess RRA
Questions from attendees concerned the formulation of the definitions in the Dimensions Report and the case studies. Initial ideas for the definitions were gathered from a panel of GRC members and then expanded iteratively over time. It is hoped to have a minimum of three case studies representing each of the dimensions and coming from all 5 geographical regions. DORA is keen to help promote the GRC case studies and can contribute updated case studies from the website.
The slides from the presentation are available here: