Guest post by Natalia Gras and Judith Sutz
In this blog post, Natalia Gras and Judith Sutz examine the complex journey to reform research evaluation in Uruguay, tracing how a metric‑driven system has shaped academic life and how collective efforts are now opening possibilities for change. Drawing on extensive research, national consultation, and policy engagement, the authors reflect on the challenges of transforming research assessment in a peripheral academic context while advancing principles aligned with DORA: qualitative judgment, recognition of diverse contributions, and socially relevant knowledge production.
From Quantification to Invisibility: The Costs of Metric‑Driven Evaluation
In Uruguay, as everywhere else, the New Public Management approach to policy conditioned the way research assessment was conducted. The quantitative imperative transformed activities into numbers, and those activities that were difficult or even impossible to measure became “non-activities”, invisible for the assessment exercise, even if they may take a good deal of researchers’ time and energy.
The Uruguayan research assessment landscape, like that of other peripheral countries, faces a main issue: the granting of academic legitimacy is overwhelmingly concentrated in the core of highly industrialized countries. The label of prestige is attributed to their academic institutions and their validating tools; following the latter to the letter has been perceived by academic elites in the periphery as the way to earn that coveted label. As Altbach (2003) put it: “[T]he Third World looks to the North for validation of academic quality and respectability. For example, academics are expected to publish in reputable academic journals within their respective disciplines. Promotion often depends on such publication. Even where local scholarly publications exist, they are not respected.” (Altbach 2003: 6). Conservatism in academic assessment is a general feature, but the point we want to make is that fighting it in the periphery needs to struggle with a higher degree of conservatism, something like “let’s become as they first, and then we will think of changing”.
The following reflection, coming from Chile (Labraña et al, 2026: 17), more than 20 years after the former, confirms the resilience of the problem:
This affective adherence to metrics becomes particularly striking when viewed in a global context. At a time when higher education systems in the Global North are increasingly questioning the validity of metric-driven governance, as evidenced by initiatives like the Coalition for Advancing Research Assessment (CoARA), Chilean higher education appears to be doubling down on quantification (…) This divergence suggests a center–periphery dynamic in which “modernization” is an affective aspiration, where legitimacy is equated with the adoption of evaluative technologies that are already being critiqued or dismantled at the center.
This forms one basic feature of the background: the greater difficulty of changing, which is always present, due to the peripheral condition.
To that should be added the “loneliness” of academia in peripheral countries: neither industry nor government is a strong source of cognitive demand for it. This tends to push researchers into self-defined research agendas, a trend reinforced by a research assessment system that does not value much collaboration with external actors. As an idiosyncratic point, Uruguay has an academic system where a single institution, the University of the Republic (UR), concentrates over 75% of the researchers and the research done in the country; in this institution, undergraduate teaching is mandatory, but academic prestige comes from research, fueling a sort of academic schizophrenia with negative effects on the researchers’ quality of life.
Early Resistance and Missed Turning Points
Uruguay suffered under military rule from 1973 to 1984, during which the country’s academic life came almost to a halt, and a massive exodus of researchers occurred. Once democracy was reestablished, a UNESCO-sponsored program helped rebuild the exact and natural sciences and the nascent computer sciences. Quantitative criteria dominated the program’s research evaluation. In 2004, a small group of its young scholars revolted against counting paper and using the journals’ impact factor indexes, underlining their deleterious effects on academic performance and ethics. They put forward arguments that can be found in DORA’s positions. Their proposal to change the program’s research evaluation was formally approved. Changes, however, did not follow.
In 2007, a national system for classifying and evaluating researchers, heavily based on quantitative measures, was established; undergraduate teaching was not considered. Working with non-academic actors was also neglected. Producing papers in high-impact factor journals was the hallmark. The National System of Researchers (NSR) established a “legitimacy regime”: researchers were those, and only those, admitted to the system.
In 2012, the UR proposed the establishment of an explicitly qualitative perspective, again, quite near to DORA’s approach, when assessing the academic performance of full-time faculty. It was accepted, but given the NSR prioritizations, dedication to teaching, outreach, and working on problems with little immediate academic return was felt as a burden.
In 2018, the UR organized a meeting on research evaluation to share the findings of a growing international critical literature on the issue. More than 300 researchers attended, and lively discussions took place in several workshops after the plenary expositions and exchanges. A strange situation emerged: those in favor of changes were outspoken, but those against shied away. Thus, the need to better know the researchers’ opinions on research evaluation took force.
When Discontent Becomes Collective: Evidence, Debate, and Mobilization
In 2022, an interdisciplinary research group from the UR (CiTINDe, Science, Technology, and Innovation for a New Development) conducted a national survey to gather researchers’ opinions, including a wide array of questions on research evaluation. The results reflected a wide and deep longing for change, but at the same time showed how fragmented the research community was, along lines of gender, seniority, and discipline (Sutz y Gras, 2024).
Two broad and strong consensus among researchers: it is necessary to transform the national evaluation system to (i) discourage bad practices (86.1%) and (ii) improve the link between knowledge production and its socially relevant use (82.9%). Although with lower levels of consensus, a clear majority (66.7%) agree that if evaluation were more qualitative, they would devote greater efforts to the creative pursuit of other academic activities (e.g., pedagogical innovation, new forms of engagement with society, etc.). This opinion is stronger among early-career researchers and women and weakens considerably among those at the highest levels of the NSR and men.
The same split of opinions occurs with the perceived effects of evaluation on researchers’ mental health. A considerable majority (61.7%) agree that the current evaluation criteria cause them anxiety, with this perception being more prevalent among early-career researchers and women; the opposite is true among men and those in the highest positions in the academic system.
The aspect that most divides Uruguayan researchers’ positions is the assertion that articles published in certain journals are a good quality proxy. Although the majority (52.8%) indicate that this is not a good proxy, a significant proportion (43.9%) consider the opposite to be true. At the highest level of the NSR, the adequacy of this measure strongly prevails (66.7%).
These results were presented in August 2023, during the seminar “STI Policies in Uruguay: Researchers’ Opinions” which, between those present and those who followed by streaming, reached an audience of around 1,500 people from different institutions of the national scientific and technological system. There, the Rector of the UR called for the creation of an Interinstitutional Group (IG) to address the challenges posed by research evaluation and to discuss the prospects and characteristics of its possible transformation at a national level.
Thus, during 2024, the University Research Council, its Academic Unit, and CiTINDe, organized a series of seminars, “Evaluation is necessary: inter-institutional meetings on academic evaluation”, to discuss why, what, and how to evaluate, making available, before each meeting, texts describing international discussions and experiences.
The three face-to-face seminars were attended by a vast audience from almost all institutions of the country’s STI system. All participants agreed that research evaluation was important for expanding and strengthening the national academic community, improving its practices, and contributing to the production of high-quality, original, relevant, and socially pertinent knowledge. The need for transformations emerged during the discussions. Evaluation needs to become more qualitative than quantitative; all academic activities (not only research) need to be assessed; the diversity of the academic’s production and its traditions and practices need to be acknowledged; processes and activities, not only products, need to be considered; last, but not least, the context in which academic work is carried out needs to take part in the evaluation, along a gender perspective.
These discussions and “proto-agreements” paved the way for the concretization of an Interinstitutional Group to explore in greater detail how the national research evaluation system could be modified. For the first time in Uruguay, public and private universities, public research institutes, researchers’ organizations, the NSR, and the National Agency for Research and Innovation sit around a table to discuss the matter. After several meetings, an agreement was reached on fifteen points that describe how research evaluation should be approached at the national level by all institutions, leaving ample room for implementation for each of them (“Fifteen agreements for academic assessment in Uruguay”). DORA’s materials were of paramount importance in guiding discussions and legitimizing the need for change.
For the first time in Uruguay, public and private universities, public research institutes, researchers’ organizations, the National System of Researchers, and the National Agency for Research and Innovation sit around a table to discuss the matter.
The Fifteen Agreements: Toward a Shared Framework for Fair and Context‑Sensitive Evaluation
The IG agreed that the central aim is to use evaluation as a valuable tool for building a robust and dynamic national academic community. This community should produce high-quality advanced knowledge, enable its dissemination, and promote its socially relevant application (Agreement I). The goal is also to expand and foster the healthy collective development of the country’s STI system (Agreement III).
More than two-thirds of the members of the national academic community are assessed both by their institutions and the NSR. In light of the tensions this creates, it was agreed that it is imperative to make progress towards convergence or harmonization between each institution’s assessment criteria and the national criteria established by the NSR (Agreement II). It aims to contribute to the achievement of institutional and national development objectives, as well as to fulfill the academic responsibilities and activities required by them. As already mentioned, the vast majority (more than three-quarters) of researchers carry out their activities at universities, particularly the UR.
It is imperative to make progress towards convergence or harmonization between each institution’s assessment criteria and the national criteria established by the National System of Researchers.
It follows the importance of considering and assessing the full range of academic activities (not just research) required by each institution. Particular care should be taken not to overlook undergraduate teaching, technological output, engagement, and links with society and institutional development (Agreement X). It is worth noting that this agreement does not require all academic activities to be carried out simultaneously within the same evaluation period.
The IG acknowledged the existing disparities within the national academic community, driven by differences in terms of expansion and level of development of disciplines, geographical areas, institutions, investment, and access to research resources. Gender inequalities leading to quantitative unequal academic performance were also considered. Evaluation must take these contextual factors into account and address them to move towards a more inclusive, diverse, and robust community (Agreement VIII).
For a proper assessment of individual academic performance, additional criteria should be incorporated. (i) Recognition and valuation of the many activities that are often overlooked and are essential for ensuring the necessary conditions for carrying out research (Agreement XI). (ii) Consideration of the inherent dynamics of the processes of original knowledge creation, for not all stages produce the same types of results or outputs; therefore, evaluation must take an intertemporal perspective on research cycles (Agreement XIII). (iii) Finally, research being an essentially collective activity, individual performance evaluation must promote, rather than penalize, collaborative work (Agreement XII).
In Uruguay, the assessment system is overburdened, placing a disproportionate strain on institutions, those being assessed, and assessors alike, jeopardizing the quality of its results, wasting valuable opportunities for learning and improvement, and, in many cases, causing discouragement and frustration among those being assessed. Better evaluation requires reducing the frequency of evaluations and dedicating time to conducting high-quality evaluations and providing appropriate feedback that contributes to strengthening the quality of future academic activities (Agreements IV, V, VI, VII, and IX).
Finally, the IG agreed that, to transform the evaluation process, it is essential to diversify its sources of information. The development of narrative CVs was agreed upon to appreciate the diversity of activities and outcomes, as well as the academic significance and social relevance of these contributions (Agreements XIV and XV).
The Future: Will Change Take Root?
These agreements were achieved, will actions follow? One important point is in the making: translating the most substantive agreements into the categories of a narrative module to appear in the widely utilized online CV model. Some agreements have already been achieved on the content: (i) it will be optional to fill; (ii) it will have five fields, with a short description for each, and a maximum of 250 words allowed.
The UR team proposed the following content for the narrative module:
Field 1.- Contribution to Knowledge Generation
Describe your main contributions to knowledge generation, new ideas, tools, and/or methodologies, highlighting their significance (novelty, production of relevant evidence, originality, contributions to public policy, or any other element that helps to assess their scope and importance; if applicable, include the process of constructing the object of study).
Field 2.- Contribution to the training of individuals and groups
Describe your contribution to the training and academic development of others – teaching activities, mentoring, or thesis supervision – and/or the creation, management, or coordination of work teams.
Field 3.- Contribution to the science, technology, and innovation system and community
Describe your role and results in relevant activities related to academic management or leadership, evaluation (of individuals, projects, theses, publications, etc.), creation or strengthening of institutions, networks, scientific societies, and any other activity that requires significant dedication and contributes to the proper functioning and consolidation of the science, technology, and innovation system and community.
Field 4.- Contribution to society and main applications of the knowledge produced
Describe the main contributions of your academic work in different spheres of social activity—productive, educational, cultural, public policy, informed citizen debate—and other relevant real or potential applications and impacts in social or productive sectors. Select between three and five keywords that describe the spheres of social activity to which your academic work contributes or may contribute. You will be expressly asked for authorization from the National Agency for Research and Innovation to build a database using this field to make better known the national research contributions.
Field 5.- Contextual Information
If you deem it necessary, describe personal, institutional, or current situations that allow for a better appreciation of your activities and results.
The National Agency for Research and Innovation has now accepted -with minor modifications to that proposal- a narrative module for the national online CV, a concrete outcome of the collective work nurtured within the IG, which has become a community of practice for those involved in the topic of research assessment in Uruguay.
The new module opens space for researchers to describe, in their own words, not only what they produce, but how and why their work matters: across knowledge creation, teaching and mentoring, institutional and community contributions, social engagement, and the often‑ignored contexts in which academic work unfolds. This is an important step forward, but it is not the end of the story. Optional fields can easily become invisible fields. The real question remains: will this space be used, and by whom? Experience suggests that without collective support, narrative sections risk being perceived as yet another administrative burden.
This is precisely where we now turns our attention: working with small groups, departments, and early‑career researchers to turn this tool into a shared response to academic discomfort, inequality, and misrecognition. Acceptance is an important milestone to be celebrated. But making the narrative CV meaningful in practice is the next collective challenge.
References
Altbach, P. (2003) (ed.) The Decline of the Guru. The Academic Profession in the Third World, N. York: Palgrave Macmillan
Labraña, J., Brunner, J.J., Rodriguez‑Ponce, E., Gómez, M.R., and Wee, C. (2026) “Quantification as Institutional Narrative and Affective Device: Reframing University Community Engagement in Chilean Higher Education Policy”. Higher Education Policy. https://doi.org/10.1057/s41307-025-00436-w
Sutz, J., & Gras, N. (2024). La evaluación de la investigación: no cambiar, cambiar, cómo cambiar. Integración Y Conocimiento, 13(1), 109–135. https://doi.org/10.61203/2347-0658.v13.n1.44216