There certainly is not a magic bullet when it comes to comprehensive and efficient research assessment, whether in the humanities or STEM fields. Publisher prestige currently influences tenure assessments in the humanities, as do journal names in the life sciences.
On Wednesday, April 3, we’ll be talking to Janet Halliwell, a member of the Impacts Project Advisory Group of the Canadian Federation for the Humanities and Social Sciences about assessment in the social sciences and humanities. She contributed to the report, “Approaches to Assessing Impacts in the Humanities and Social Sciences” which provides straightforward examples of how different impacts from diverse areas of scholarship can be traced and assessed.
Academic outputs are distinguished from academic and societal impacts in the 2017 report. By separating outputs from impacts, we can answer the question “What are we expecting the researcher to achieve through their work?” In the case of academic impacts, the report explains, if we want a scholar to contribute to academic knowledge, then we should look at scholarly outputs (such as articles or books written or conference talks given). On the other hand, if we want a researcher to contribute to capacity-building in academic culture, then indicators of their ability to cultivate talent (assessments of lectures, mentee’s outcomes) should be emphasized in evaluations. The measures we choose should appropriately assess the activities and outcomes we want. Appropriate measures can serve as incentives for building the academic culture we want.
Designing academic assessment procedures which use indicators that align with institutional goals can be a useful way for research administrators in any field to redefine assessment criteria to be fit for purpose.
But what about impacts on society? Often scholars in the humanities and social sciences are asked to justify their scholarship by demonstrating impacts on policy, society or the economy. But these connections can be difficult to trace, as the impacts of these forms of scholarship can be felt broadly across society, in both direct and indirect ways, according to the report.
Case studies, such as those described in the report, can illustrate the indirect impacts of scholarship, and help describe indirect or long-term impacts in research assessment procedures. One example offered is a description of the impact of a piece of folklore scholarship. In this example, an expert in Eastern-European folklore analyzes and writes a book about how linguistic phrases can be associated with particular territories in a traditional Romanian folk tale. Decades later, this scholarship serves as a key piece of evidence in a case brought to court on behalf of a small ethnic group in Romania. The scholar’s analysis was used in validating a centuries-old territorial claim of the ethnic group, and resulted in their winning the court case. The scholar, however, never learns about how this discovery was used, making it impossible for this impact to be reported in an assessment.
If scholars themselves do not always know about the impacts of their work, what can we do? Join us on Wednesday, 11 EST, to discuss this issue and more, as we talk to Janet Halliwell about creative approaches to assessment in the social sciences and humanities.
Helen Sitar is a Science Policy Programme Officer at EMBO