DORA is committed to supporting the development of new policies and practices for responsible research assessment. This tool compiles searchable criteria, hiring standards, review, promotion, and tenure from academic institutions.

Reformscape Glossary

 

 

 

Types of source material

 

Announcement A statement or press release from an academic institution saying they have signed up to or endorsed a responsible research assessment campaign or initiative, such as DORA, the Leiden Manifesto or Hong Kong Principles on Research Integrity.
Action plan Document that describes plans to investigate and implement responsible research assessment (e.g., an action plan, road map, framework, etc.).
Principles Document created by an academic institution that articulates its own principles to promote responsible research assessment (e.g., uses of metrics). These are often informed by guidance set out by DORA, the Leiden Manifesto, and/or the Metric Tide Report.
Policies Descriptions of standardized, but generalized, processes or performance criteria formalized in an organization’s guidelines and procedures (e.g., faculty handbook).
Practices Specific and/or concrete behaviors or examples in support of responsible research assessment, which may be contextualized interpretations of codified policy or emerge independently of formal policy documents.
Outcomes The results that have emerged from given assessment reforms.

 

 

Types of decisions that impact research careers

 

Hiring The process of selecting and employing academic faculty and/or staff.
Promotion The process through which academic faculty and/or staff advance to a higher position within the institution.
Tenure The process of applying for tenure defined as an indefinite academic appointment that can be terminated only for cause or under extraordinary circumstances. Adapted from: https://www. aaup.org/issues/tenure
Retention The process of improving the ability of academic institutions to retain faculty and staff.

 

 

Assessment topic areas

 

Open research An inclusive construct that combines various movements and practices aiming to make multilingual scholarly knowledge openly available, accessible and reusable for everyone, to increase scientific collaborations and sharing of information for the benefits of science and society, and to open the processes of scholarly knowledge creation, evaluation and communication to societal actors beyond the traditional scientific community.
Adapted from: https://en.unesco.org/science-sustainable-future/open-science/recommendation, The UNESCO Recommendation on Open Science (2021, p.7)
Research integrity Behavior that leads to trustworthy research by avoiding questionable research practices.
Definition from: https://journals.plos.org/plosbiology/article?id=10. 1371/journal.pbio.3000737, The Hong Kong Principles for Assessing Researchers (2020)
Principles Document created by an academic institution that articulates its own principles to promote responsible research assessment (e.g. uses of metrics). These are often informed by guidance set out by DORA, the Leiden Manifesto, and/or the Metric Tide Report.
Academic contributions The outcomes, outputs, and value generated by an individual’s academic contributions.
Societal impact The effect of scholarly work beyond academia to the economy, society, culture, public policy or services, health, and the environment or quality of life.
Adapted from: https://www.purdue.edu/engagement/research-impacts/
Cultivating talents The results of investments such as mentoring, teaching, or career guidance provided to early career or emergent individuals, which manifests as success either inside or outside of academia.
— Service Contributions to faculty administration, institutional policy, student affairs, leadership and/or contributions to the broader research community.
Adapted from: https://www.tandfonline.com/doi/abs/10.1080/13600800701457863?journalCode=cjhe20
— Entrepreneurship Activities relating to commercialization, industry collaboration, and/or wealth creation.
— Research collaboration Teamwork among researchers and across disciplinary, organizational, and cultural boundaries to address increasingly complex challenges and opportunities in science and society.
Definition from: https://psycnet.apa.org/record/2018-23205-018
— Academic impact The demonstrable contribution that academic work makes in shifting understanding and advancing scholarly method, theory and application across and within disciplines.
Adapted from:
https://www.ukri.org/councils/esrc/impact- toolkit-for-economic-and-social- sciences/defining-impact/
Responsible metrics Fair and appropriate use of metrics in academic assessments.
Equity and inclusion Effort to curb discriminatory practices and behaviors, or encourage inclusive practices and behaviors, to achieve parity of researcher career patterns, regardless of race, ethnicity, orientation, gender, or other immutable characteristics related to identity
Adapted from: https://cue.usc.edu/equity/
Portfolio assessment Multidimensional assessment criteria are used to assess a candidate’s merits – often contrasted with overreliance on narrow range of traditional indicators of merit (e.g. publication output, citations, funding revenue).

 

 

Academic disciplines

 

Natural sciences E.g., biology, chemistry, physics
Design and engineering E.g., architecture, structural engineering
Social sciences E.g., sociology, political science
Arts and humanities E.g., music, theatre
Medical sciences E.g., nursing, physician assistant
Discipline non-specified Unknown
Cross-disciplinary One more more of above

 

 

Populations

 

Faculty A group of academic personnel, which can include assistant professors, associate professors, full professors, as well as those who are primarily engaged in teaching or research (i.e. teaching or research faculty) or considered to be adjunct or visiting faculty.
— Assistant professor Entry-level position within academic institutes. This position typically involves research, teaching, and administrative responsibilities.
— Associate professor An intermediate academic position and a more senior level than assistant professor. Duties are largely the same as for assistant professor, but expected to come with more intensive demands and higher salary.
— Full professor Highest academic position held at an academic institution. They are typically judged to have excelled in certain academic contributions (research, teaching, and/or service related) and may take on further responsibilities than associate professors.
— Other faculty types Types of faculty members that are not considered to be Assistant, Associate, or Full Professors.
— Population non- specified Staff from across multiple roles and levels of seniority are implicated in the intervention.
All staff  

 

 

Types of institution

 

Comprehensive university or equivalent An institution that provides research and teaching covering multiple academic disciplines.
Specialized university or equivalent An institution providing research and education predominantly within a specialist field or discipline (e.g. medical science, music and arts schools).
University of applied sciences A college-type or professional education institution which does not award PhDs, or does so in only a few disciplines and is oriented primarily around practical knowledge.
Non-university research institute

Stand-alone, independent organizations primarily undertaking scientific research with dedicated streams of state funding. Definition from: https://doi.org/10.1007/s11024-019-09393-2

The TARA team adapted this term, which in the original reference is called ‘Extra-university research institutes’. They can be connected via large umbrella associations or coordinated by government agencies (Powell et al 2017). Examples include German Fraunhofers, or French CNRS.

Distance learning university An institution where students enroll for a degree study principally from home or work.

 

 

Scope of policies

 

Department An academic unit of an institution dedicated to a particular academic discipline(s).
School/division An institution providing research and education predominantly within a specialist field or discipline (e.g. medical science, music and arts schools).
Institution-wide An entire institution.

 

 

Arcadia is a charitable foundation that works to protect nature, preserve cultural heritage and promote open access to knowledge.

For questions related to Reformscape: reformscape@sfdora.org