A DORAat10 Local Event Report
In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.
By Stephanie Warner, PhD, Manager, Knowledge Engagement, University of Calgary with support from representatives of NSERC, SSHRC, CIHR, Genome Canada and CFI
To remove barriers to research funding, hiring, tenure and promotion that many researchers in Canada continue to experience, sustained and collaborative effort is needed. The Declaration on Research Assessment aims for thoughtful inclusion of broader types of research outputs and societal impact in assessment of researchers for funding, tenure, promotion, merit and hiring.
The DORA movement in Canada is growing, with 47 organizations having signed the commitment to more effective and robust approaches to research assessment. The five major Government of Canada research funders (Social Sciences and Humanities Research Council/SSHRC, Canadian Institutes of Health Research/CIHR, Natural Sciences and Engineering Research Council/NSERC, Genome Canada and Canada Foundation for Innovation/CFI) all signed DORA in 2019. Although many institutions in Canada are adopting principles related to responsible research assessment and research impact, only seven Canadian postsecondary institutions have signed DORA.
The University of Calgary was the first university in Canada to sign DORA in 2021, and is committed to advancing conversations, practices and policies that reflect the diversity and priorities of our research community.
The Canadian Conference on Research Administration
The Canadian Association of Research Administrators annual conference brings together funders and postsecondary research administrators – the individuals who support researchers to navigate funding, ethics, legal contracts, equity, diversity, inclusion and accessibility (EDIA) in research, community engagement and research impact – to discuss updates, expectations, and experiences. This meeting is one of the few existing venues within the research ecosystem for deep conversations about research excellence and culture change.
DORA leaders from the University of Calgary, Université de Montreal, NSERC, SSHRC, CIHR, CFI and Genome Canada co-developed a half-day workshop to raise awareness of what DORA is and what it means for assessing research excellence, provide updates on the implementation of DORA into organizational and funding practices in Canada, and enable opportunities for discussion and idea generation among all participants.
The workshop
The session room was full on May 14, 2023, with 49 participants. Their interest and depth of engagement showed that the time is right for deeper engagement between higher education and funders. Through polling, we learned that around 80% of participants were already familiar with DORA, and 66% said their institutions were aware of DORA (only half of those institutions had signed DORA). When asked if their organization is currently working toward implementing DORA and responsible research assessment practices, 40% said yes while 52% were unsure.
The session opened with an overview of DORA from its Acting Director Haley Hazlett. Vincent Larivière, Université de Montréal, spoke on the evolution of research assessment and the responsible use of metrics, followed by lightning updates from the University of Calgary, CIHR, NSERC, SSHRC, and Genome Canada (view slides).
The second half of the workshop allowed attendees to rotate in groups through four discussion questions:
- How do you feel the recommendations of DORA will change the definition of research excellence?
- What support do you need from funding agencies to make this change?
- What support do you need from your institution to make this change?
- What are your suggestions for overcoming barriers?
While each table had areas of focus that emerged, there were also a number of overarching themes and take-aways.
Equity, Diversity, Inclusion and Accessibility
Overall, attendees pointed to DORA opening a broader set of possibilities under the umbrella of research excellence and understanding of different value systems such as Indigenous and community-first values. Emphasis on the differences in disciplinary norms and ways of working, as well as values emphasized in different communities, points to the relative ease of change in some fields. Examples of holistic and more qualitative impacts from these fields may help to highlight and visualize the positive impact of DORA-aligned assessment practices. Overall, attendees felt that DORA would lead to a more diverse pool of people deemed successful.
Who is responsible for leading the movement?
Everyone is looking for a leader – be it institutional leaders/executives, research councils, or other organizations – to set the expectations firmly and clearly in a way that is easy for others to follow. We were surprised by the frequency with which attendees mentioned the lack of leadership-level commitment to DORA and feel that this requires further discussion. For DORA to take shape in a complex multi-dimensional system, change and commitment is required across all levels.
What can shift the ecosystem to a new paradigm?
Awareness, education and resources arose as key levers for change. Many funders are updating guidance for merit review and reviewer guidelines. To supplement this, participants suggested that funders provide short, easy-to-read resources with clear guidance for both applicants and reviewers (for example, DO NOT include Journal Impact Factor or h-index in applications; reviewers SHOULD NOT consider these if included). Proper training and education of review committees will be crucial to support integration of these policies into practice.
Institutions could dedicate more time and personnel to raising awareness and look to leadership to sign on to DORA. Obviously, any discussion of research assessment must also include the researchers themselves. Attendees planned to return and have those conversations at their own institutions.
Next steps
DORA encompasses more than we may think initially. In Canada, DORA can be a complementary approach to strategic goals, such as equity, diversity, inclusion and accessibility (EDIA), Open Research/Science, and doing Indigenous research in a good way. Together, we must continue the conversation, sharing the opportunities and “wins” that DORA-aligned research assessment affords and the challenges that emerge along the way.
The organizers of this session are committed to further unpacking what we heard, and working together on practical resources, outputs and engagement opportunities that will benefit the Canadian research community.
