Background: The Metrics Literacies project
The Scholarly Communications Lab’s Metrics Literacies project, funded by the Social Sciences Research Council (SSHRC) of Canada, aimed to test the effectiveness of different multimedia formats for educating researchers about scholarly metrics. Members of the research team have previously established the concept of metrics literacies, defined as:
“an integrated set of competencies, dispositions, and knowledge that empowers academics and research administrators to recognize, interpret, critically assess, and effectively and ethically use scholarly metrics” (Maggio et al., 2022, p.16).
Educational videos were produced as research instruments in a randomized controlled trial (Haustein et al., 2025a; Haustein et al., 2025b). Using the h-index, a widely known and popular but controversial indicator, as a case study, the research team developed learning objectives, which were used to inform the learning content incorporated in video scripts. The videos’ learning objectives were as follows:
1. Popularity, use and misuse, optimizing metrics and gaming
This objective, used to provide context, aimed to help viewers state the reasons why the h-index has become so popular in academia, including for hiring, promotion, and the allocation of funding and awards. They should also recognize the uses and misuses of the h-index and identify questionable research practices and strategies academics use to optimize and game the h-index.
2. Definition and computation
Viewers should be able to list the components of the h-index definition (i.e., publications and citations), compute the h-index, and describe how the number of publications can be a limiting factor in how the h-index is defined.
3. Inability to assess early career researchers
Under this objective, learners should be able to recall the effect of career length on the number of publications and the h-index. Following from this, they should be able to describe the h-index’s inability to assess early career researchers.
4. Lack of field normalization
Differences in publication practices that affect the h-index across research fields should be identified. Viewers should also identify that, because of these varying publication practices, the h-index cannot be used to compare researchers from different fields.
5. Which h-index?
Finally, viewers should recognize how the h-index can vary for individual researchers, depending on when it was calculated and which database was used. They should also be able to identify factors, such as indexing policies, that cause the h-index to vary between databases.
The same learning objectives were addressed in a briefing note on the h-index (Haustein et al., 2025c), which represents the traditional, written means of scholarly communication and includes the same content as the video scripts.
Educational videos
Development
The research team first conducted a video taxonomy literature review (Haustein et al., 2025d) from which the talking head and animation formats were selected for their proven effectiveness in educational settings and human elements that engage audiences. For example, talking head videos leverage the presence of a narrator who delivers a lecture-style presentation, often accompanied by slides presenting the content they speak over. The narrator guides learners through learning material, emphasizing gestures, tone, and facial expressions to foster social connection and authenticity. Animation, on the other hand, leans more on storytelling and multimodal visual and audio, a combination proven to enhance viewers’ retention of information.
The following stills of the educational videos on the h-index show the same content presented in animation format (left) and talking head format (right). The image below depicts each video teaching about how the h-index is defined and calculated, fulfilling learning objective 2: Definition and Computation.
The next example presents content educating viewers on the h-index’s lack of field normalization, meeting learning objective 4.
The research team also developed personas to reflect real-world academic scenarios (Dorsch et al., 2020). Two personas were selected for use in the educational videos. The first, Dr. Maryam Hashemi, is pictured in an animated video still. Maryam is a postdoctoral researcher at a university in Iran. She has a PhD in biotechnology and is looking for a tenure-track position in a biology department. She knows that in her field, the h-index plays an important role in hiring and promotion, and is therefore debating whether or not she should include it in her CV when applying for a position.
The second persona is Dr. George Clark, pictured atop a paper boat in a frame from the animation video. George is a full professor at the Department of English Literature at a university in the United Kingdom. George is currently applying for his university’s Researcher of the Year award, which is a prestigious prize and comes with research funding. George was encouraged by his colleagues, who thought he would have a great shot at winning since he recently received recognition from the Royal Society of Literature. His faculty’s research office was also happy to hear that he wants to apply and is helping him fill out the application package, which asks for his number of publications and his h-index.
A script incorporating these personas was crafted for each video, along with accompanying slides for the talking head video. Video and narration were recorded at the University of Ottawa Morisset Library’s podcast studio with the technical support of the institution’s active learning technologist. A theatre student at the university acted as the talking head and recorded narration for both videos. An animation student and instructor at Concordia University developed the storyboard, illustrated the animation, provided sound effects, and produced the animated video. The Metrics Literacies research team produced the talking head video using WeVideo editing software. Although structured differently, the videos deliver the same content and are both under 7 minutes in length. They cater to different learning preferences by leveraging distinct formats, offering an engaging and informative experience.
Potential uses and implications for metrics education
Video format suggests itself as an (open) educational resource alternative to the traditional written text with immense pedagogical potential. Creative means of presenting educational material may offer more accessible ways for researchers and the academic community to increase their awareness and critical use of commonly used scholarly metrics.
As the future of the research assessment reform movement continues to evolve, finding innovative ways to teach students, faculty, staff, and administrators about scholarly metrics is vital. Their limitations and implications for research evaluation must be clearly conveyed. Educational tools such as the two videos on the h-index presented above can facilitate learning and encourage critical thinking about scholarly metrics. More importantly, community-driven education materials development tends to place greater emphasis on metrics literacies—helping individuals understand the strengths, limitations, and appropriate use of metrics—compared to commercially developed resources, which often focus primarily on the mechanics of evaluation itself.
The bibliometrics community has a wealth of knowledge and skills to share regarding metrics education and guidance. Fostering meaningful change requires creative approaches and meeting learners where they are. DORA’s Reformscape tool and Resource Library is one approach– scholarly materials (including the Metrics Literacies outputs) developed to inform or conduct research evaluation are made openly available. By bridging the gap between scholarly research and evaluation practices, Reformscape offers a pathway toward a future of more informed and nuanced assessment approaches.
Other educational materials
The Metrics Literacies Zenodo Community hosts all outputs of the project, including the videos, in a citable format (Haustein et al., 2025a,b). The animated video and talking head video are also available on YouTube. All educational materials are attributed a Creative Commons CC-BY Share Alike 4.0 International license. This permits almost any use (e.g., sharing, revising, remixing, reusing), as long as the creators are credited. We encourage the widespread use and application of these materials in teaching and research. In response to several requests to translate the materials, interested parties are welcome to use the script and get in touch with Madelaine Hare or Stefanie Haustein for high-quality audio and visual files.
Acknowledgments
The authors would like to thank Marie-Josée Archambault, Anthony Artino, Germana Barata, Robin Champieux, DeDe Dawson, Jonathan Degan, Julie-Ann Déry, Alice Fleerackers, Elizabeth Gadd, Fiona Smith Hale, Alyssa Jeffrey, Sam Kennifick, Sarah Komendat, Lauren Maggio, Tristan Lamonica, Peter Musser, Sanam Ebrahimzadeh Pirshahid, Michelle Reidlinger, Sarah Van Sickle, and Alli Torban for their immense contributions to the project from inception in 2018, grant writing, and the multiple stages of research and video production.
References
Dorsch, I., Ebrahimzadeh, S., Jeffrey, A., & Haustein, S. (2020). Metrics Literacies: Introduction of researcher personas for the understanding and use of scholarly metrics (Version v1). Zenodo. https://doi.org/10.5281/zenodo.4046019
Haustein, S., Hare, M., Dorsch, I., Chen, C. M.-L., Woods, H., Déry, J.-A., & Archambault, M.-J. (2025a, March 5). What is the h-index and what are its limitations? Or: Stop using the h-index [Animation video]. Zenodo. https://doi.org/10.5281/zenodo.14975364
Haustein, S., Hare, M., Dorsch, I., Chen, C. M.-L., & Woods, H. (2025b, March 5). What is the h-index and what are its limitations? Or: Stop using the h-index [Talking Head video]. Zenodo. https://doi.org/10.5281/zenodo.14975464
Haustein, S., Hare, M., Dorsch, I., Chen, C. M.-L., & Woods, H. (2025c). H-index briefing note. Zenodo. https://doi.org/10.5281/zenodo.14975485
Haustein, S., Hare, M., Dorsch, I., Chen, C. M.-L., & Woods, H. (2025d). Video taxonomy literature review. Zenodo. https://doi.org/10.5281/zenodo.14975503
Maggio, L. A., Jeffrey, A., Haustein, S., & Samuel, A. (2022). Becoming metrics literate: An analysis of brief videos that teach about the h-index. PLOS ONE, 17(5), e0268110. https://doi.org/10.1371/journal.pone.0268110