Return to the course homepage. This resource is part of the supporting materials in the “Materials” tab on the course homepage, where you can find other resources like How to Navigate the Course and the Teaching Support Guide.
Glossary
- Research assessment is the process used to review researchers’ scholarly contributions, evaluating their quality, value, and relevance, which influences aspects of academia such as hiring, promotion, funding decisions, and science policy. Assessment occurs at multiple levels of the academic system, including individual researchers, research groups or units, entire institutions (such as universities), or national research systems, at different moments in time. It encompasses a system of measurements and reviews that determines how scholarly labor is categorized and rewarded within the institutional framework of science and higher education institutions.
- Responsible research assessment (RRA) is a modern, holistic approach to evaluating research and researchers that incentivizes and rewards diverse, high-quality contributions and their real-world impact, moving beyond narrow quantitative metrics to promote fairness and transparency in support of diverse and inclusive research cultures.
- Journal Impact Factor (JIF) is generally defined as the annual average number of citations to papers published in a given journal in the two preceding years.
- h-index for individual authors is defined as the *h* number of their papers that have been cited at least h times, and is calculated based on a researcher’s number of publications and citations.
- Citations are defined as the number of times an article, book, policy, or alt-metric is included in the reference list of other scholarly works.
- Metrics are quantitative numbers often used as proxy measures of quality, performance, and/or impact in research assessment, though the term “indicator” more precisely reflects their indirect nature.
- Indicators are quantities used in research assessment that more accurately reflect their role as indirect proxies for quality rather than direct measurements.
- Bibliometrics is a quantitative method involving citation and content analysis for scholarly journals, books, and researchers, used to appraise the quantitative impact or influence of a given work on academic literature by measuring its citations.
- Reproducibility is a movement that advocates for rigor in research design and performance, transparent sharing of methods and results, and the ability to reproduce research findings.
- Open scholarship, also known as open science, is an inclusive movement aiming to make multilingual scientific knowledge openly available, accessible, and reusable for everyone to increase scientific collaborations, share information, and open knowledge creation processes to broader societal actors.
Suggested Reading List
Learn about responsible research assessment (RRA)
- The Declaration on Research Assessment (DORA)
- Link: https://sfdora.org/
- The Latin American Forum on Research Assessment (CLACSO-FOLEC)
- The Coalition for Advancing Research Assessment (CoARA)
- Link: https://www.coara.org/
Learn about metrics and indicators
- “Guidance on the responsible use of quantitative indicators in research assessment” by DORA (2024)
- This guidance analyzes commonly used quantitative indicators, emphasizing their limitations when applied to research assessment.
- Link: https://zenodo.org/records/13844873
- Metrics Toolkit
- The Metrics Toolkit is mentioned as a resource that can help individuals better understand what information different metrics can and cannot provide.
- Link: http://www.metrics-toolkit.org/
- Dorsch, I., Ebrahimzadeh, S., Jeffrey, A., & Haustein, S. (2020). Metrics Literacies: Introduction of researcher personas for the understanding and use of scholarly metrics (Version v1). Zenodo.
- This resource provides several research “personas”. These explore the negative impacts of an overreliance on quantitative indicators, like h-index, using real world examples.
- Link: https://doi.org/10.5281/zenodo.4046019
Learn about how RRA intersects with other movements
- “The intersections between DORA, open scholarship, and equity” (Hatch, Barbour, & Schmidt, 2020)
- This blog outlines how efforts to support RRA, open scholarship, and equity intersect and support each other.
- Link: https://sfdora.org/2020/08/18/the-intersections-between-dora-open-scholarship-and-equity/
- UNESCO Recommendation on Open Science
- Adopted in 2021, this Recommendation provides an international framework for open science policy and practice, recognizing disciplinary and regional differences. Its core principles, such as transparency, equality of opportunities, collaboration, and sustainability, closely align with RRA.
- Link: https://www.unesco.org/en/open-science
- FAIR data principles (Findable, Accessible, Interoperable, Reusable)
- These principles were created to make data easier to share and reuse, supporting the reproducibility movement by advocating for transparent and interoperable sharing of methods and results.
- Link: https://www.go-fair.org/fair-principles/
- EDI Caucus (EDICa)
- Established by the British Academy and UKRI, EDICa aims to build the evidence base for developing inclusive careers in the UK’s research and innovation ecosystem.
- Link: https://edicaucus.ac.uk/research-projects/