Narrative CV formats have been implemented by multiple research funding organizations, such as the Dutch National Research Council, the Swiss National Science Foundation, Luxembourg Research Council, and UKRI, in an effort to de-emphasize flawed quantitative indicators of research quality while capturing qualitative aspects of research contributions. Though multiple organizations are currently experimenting with narrative CVs, successful implementation of responsible research assessment requires an iterative approach. At DORA’s March 16, 2021, America’s, Europe, and Africa Funder meeting, Health Research Board Ireland (HRB) Programme Manager Annalisa Montesanti presented the HRB’s implementation of a narrative CV for a grant application for research career schemes and the preliminary results from a survey of applicants and panel members.
The HRB is a State Agency under the aegis of the Irish Department of Health whose purpose is to support and fund health research, including the support of research career pathways for both academic researchers and health and care practitioners. The HRB has taken multiple steps to translate DORA’s principles to their research assessment policies and practices. These steps include recognizing a variety of research outputs (e.g., research articles, databases, policy briefings), asking researchers not to reference impact factor or H-index, practicing transparency about their research assessment practices, and implementing a narrative-based CV format. Montesanti presented the format of HRB’s narrative-based CV along with preliminary findings from a series of surveys assessing the use of this CV from the perspective of applicants and reviewers.
The HRB narrative-based CV format and criteria
The HRB’s narrative-based CV format is meant to recognize the diversity of types of researchers and to provide peer and panel reviewers with more comprehensive information on the value, quality, and potential impact of the researchers’ work and their suitability and relevance to a funding scheme. The format is structured to include six sections: basic CV information (such as title, name, employment history, and ORCID iD), career breaks, contribution to knowledge, contribution to professional development of other researchers, contributions to wider research community and society, and a personal declaration.
The inclusion of career breaks enables applicants to detail reasons for career breaks (including family care responsibilities, illness, and change in job sector or discipline). This is important because it allows for time periods not spent in research to be accounted for by reviewers when assessing applicant outputs.
For the contribution to knowledge section, applicants typically list up to five peer-reviewed research funding awards and up to ten research outputs relevant to the application. Importantly, applicants are asked to briefly explain their specific contribution to each output, and the significance of the output. In this section, applicants are encouraged to include a broad range of outputs in addition to publications. Importantly, applicants were asked to omit journal impact factors and H-indexes from their lists of publications. Reviewers are specifically asked to focus on impact of the research in the field and/or in policy and/or practice in lieu of bibliometrics when considering publications and other research outputs.
For the contribution to training and development of other researchers section, applicants were asked to describe their experiences as a supervisor or mentor to researchers and other individuals of different career stages and disciplines. In this section, applicants detailed precisely how they contributed to career development of those whom they mentored.
For the contribution to the wider research community and society section, applicants were given a non-exhaustive list of examples including teaching, peer-review, communication and outreach, administrative tasks, and contribution to Open Science.
Finally, a personal declaration section was included to offer applicants a space to describe why they would be the ideal candidate for their role (e.g., lead, co-applicant) in an application. This allowed applicants to include real-world examples of their expertise and ability.
Preliminary survey results
After discussing the HRB’s narrative-based CV format, Montesanti also broke down preliminary survey results from January and February 2021. These results were from a questionnaire given to applicants and international panel reviewers for the Collaborative Doctoral Awards. Future surveys targeting international peer reviewers and also applicants and reviewers of another funding scheme will be conducted in late 2021.
Montesanti walked through the survey results, emphasizing that while these data were preliminary and from a small group of respondents, they provided useful feedback for further refining the HRB’s narrative-based CV format. The majority of applicants who responded to the survey found that the narrative CV gave a better outline of the applicants’ research experience versus a standard CV, and said that they would advise the HRB to use a similar approach in other investigator-led grant applications. Overall, Montesanti said the general feedback from applicants was positive. Key suggestions included the need to refine the word limits on different sections of the CV to make them less restrictive and the need for more examples and guidance on how to complete the narrative CV. One applicant stated that this format may render the application of a senior faculty member to be too similar to that of an early-career applicant. However, the inability to distinguish between late- and early-career applicants may be viewed as a positive outcome that reduces unconscious biases rather than a negative outcome. Applicants also suggested the utility of a single CV repository, like Brazil’s Lattes platform, to reduce time spent repeatedly reformatting the same CV for different funding organizations. The use of ORCID iDs, which many funders are integrating into their systems, might facilitate this.
Panel reviewers were also surveyed for their thoughts on the HRB narrative CV format. The majority of reviewers who responded to the survey were somewhat familiar with the recommendations of DORA. There was a mixed consensus on the ease of assessment of the narrative-based CV. While the majority of reviewers liked the structure of the narrative-based CV and felt that it was sufficient to address the application assessment criteria, they also thought that the amount and diversity of information made critical appraisal of each application difficult.
These preliminary results demonstrate the importance of using bottom-up feedback to refine and improve top-down responsible research assessment policies and practices. The HRB will continue to assess the use of a narrative-based CV in the coming year to further refine the structure as well as improve the guidelines for applicants and reviewers around how to assess it.
DORA’s funder discussion group is a community of practice that meets virtually every quarter to discuss policies and topics related to fair and responsible research assessment. If you are a public or private funder of research interested in joining the group, please reach out to DORA’s Program Director, Anna Hatch (info@dora.org). Organizations do not have to be a signatory of DORA to participate.
Haley Hazlett is DORA’s Policy Intern