Types of source material
| Announcement | A statement or press release from an academic institution saying they have signed up to or endorsed a responsible research assessment campaign or initiative, such as DORA, the Leiden Manifesto or Hong Kong Principles on Research Integrity. |
| Action plan | Document that describes plans to investigate and implement responsible research assessment (RRA), e.g. an action plan, road map, framework, etc. |
| Principles | Document created by an academic institution that articulates its own principles to promote Responsible research assessment (RRA), e.g. uses of metrics. These are often informed by guidance set out by DORA, the Leiden Manifesto, and/or the Metric Tide Report. |
| Policies | Descriptions of standardized, but generalized, processes or performance criteria formalized in an organization’s guidelines and procedures (e.g. faculty handbook). |
| Practices | Specific and/or concrete behaviors or examples in support of RRA, which may be contextualized interpretations of codified policy or emerge independently of formal policy documents. |
| Outcomes | The results that have emerged from given assessment reforms. |
Types of decisions that impact research careers
| Hiring | The process of selecting and employing academic faculty and/or staff. |
| Promotion | The process through which academic faculty and/or staff advance to a higher position within the institution. |
| Tenure | The process of applying for tenure defined as an indefinite academic appointment that can be terminated only for cause or under extraordinary circumstances. Adapted from: https://www. aaup.org/issues/tenure |
| Retention | The process of improving the ability of academic institutions to retain faculty and staff. |
Assessment topic areas
| Open research | An inclusive construct that combines various movements and practices aiming to make multilingual scholarly knowledge openly available, accessible and reusable for everyone, to increase scientifc collaborations and sharing of information for the benefts of science and society, and to open the processes of scholarly knowledge creation, evaluation and communication to societal actors beyond the traditional scientifc community. Adapted from: https://en.unesco.org/science-sustainable- future/open-science/recommendation, The UNESCO Recomendation on Open Science (2021, p.7) |
| Research integrity | Behaviour that leads to trustworthy research by avoiding questionable research practices. Definition from: https://journals.plos.org/plosbiology/article?id=10. 1371/journal.pbio.3000737, The Hong Kong Principles for Assessing Researchers (2020) |
| Principles | Document created by an academic institution that articulates its own principles to promote Responsible research assessment (RRA), e.g. uses of metrics. These are often informed by guidance set out by DORA, the Leiden Manifesto, and/or the Metric Tide Report. |
| Academic contributions | The outcomes, outputs, and value generated by an individual’s academic contributions. |
| — Societal impact | The effect of scholarly work beyond academia to the economy, society, culture, public policy or services, health, and the environment or quality of life. Adapted from: https://www.purdue. edu/engagement/research-impacts/ |
| — Cultivating talents | The results of investments such as mentoring, teaching, or career guidance provided to early career or emergent individuals, which manifests as success either inside or outside of academia. |
| — Service | Contributions to faculty administration, institutional policy, student affairs, leadership and/or contributions to the broader research community. Adapted from: https://www.tandfonline. com/doi/abs/10.1080/13600800701457863? journalCode=cjhe20 |
| — Entrepreneurship | Activities relating to commercialization, industry collaboration, and/or wealth creation. |
| — Research collaboration | Teamwork among researchers and across disciplinary, organizational, and cultural boundaries to address increasingly complex challenges and opportunities in science and society. Definition from: https://psycnet.apa. org/record/2018-23205-018 |
| — Academic impact | The demonstrable contribution that academic work makes in shifting understanding and advancing scholarly method, theory and application across and within disciplines. Adapted from: https://www.ukri.org/councils/esrc/impact- toolkit-for-economic-and-social- sciences/defining-impact/ |
| Responsible metrics | Fair and appropriate use of metrics in academic assessments. |
| Equity and inclusion | Effort to curb discriminatory practices and behaviours, or encourage inclusive practices and behaviours, to achieve parity of researcher career patterns, regardless of race, ethnicity, orientation, gender, or other immutable characteristics related to identity Adapted from: https://cue.usc.edu/equity/ |
| Portfolio assessment | Multidimensional assessment criteria are used to assess a candidate’s merits – often contrasted with overreliance on narrow range of traditional indicators of merit (e.g. publication output, citations, funding revenue). |
Academic disciplines
| Natural sciences | |
| Design and engineering | |
| Social sciences | |
| Arts and humanities | |
| Medical sciences | |
| Discipline non-specified | |
| Cross-disciplinary |
Populations
| Faculty | A group of academic personnel, which can include assistant professors, associate professors, full professors, as well as those who are primarily engaged in teaching or research (i.e. teaching or research faculty) or considered to be adjunct or visiting faculty. |
| — Assistant Professor | Entry-level position within academic institutes. This position typically involves research, teaching, and administrative responsibilities. |
| — Associate Professor | An intermediate academic position and a more senior level than assistant professor. Duties are largely the same as for assistant professor, but expected to come with more intensive demands and higher salary. |
| — Full Professor | Highest academic position held at an academic institutuion. They are typically judged to have excelled in certain academic contributions (research, teaching, and/or service related) and may take on further responsibilities than associate professors. |
| — Other Faculty Types | Types of faculty members that are not considered to be Assistant, Associate, or Full Professors. |
| — Population non- specified | Staff from across multiple roles and levels of seniority are implicated in the intervention. |
| All staff |
Types of institution
| Comprehensive university or equivalent | An institution that provides research and teaching covering multiple academic disciplines. |
| Specialized university or equivalent | An institution providing research and education predominantly within a specialist field or discipline (e.g. medical science, music and arts schools). |
| University of applied sciences | A college-type or professional education institution which does not award PhDs, or does so in only a few disciplines and is oriented primarily around practical knowledge. |
| Non-university research institute | Stand-alone, independent organizations primarily undertaking scientific research with dedicated streams of state funding. Definition from: https://doi.org/10.1007/s11024-019-09393-2 The TARA team adapted this term, which in the original reference is called ‘Extra-university research institutes’. They can be connected via large umbrella associations or coordinated by government agencies (Powell et al 2017). Examples include German Fraunhofers, or French CNRS. |
| Distance learning university | An institution where students enrol for a degree study principally from home or work. |
Scope of policies
| Department | An academic unit of an institution dedicated to a particular academic discipline(s). |
| School/division | An institution providing research and education predominantly within a specialist field or discipline (e.g. medical science, music and arts schools). |
| Institution-wide | An entire institution. |