On June 6, 2025, DORA hosted an insightful online panel discussion titled “Implementing Responsible Research Assessment in India”. Co-hosted by Dr. Rebecca Lawrence and Dr. Moumita Koley, the event aimed to foster a deeper understanding of responsible research assessment (RRA) and its potential benefits for the Indian research ecosystem. The lively discussion brought together eminent scholars and policymakers Prof. K. VijayRaghavan, Prof. S. C. Lakhotia and Dr Michael Arentoft, shedding light on the current landscape, inherent challenges, and practical pathways for reform.
The conversation highlighted several critical points regarding research assessment in India and globally:
Metric obsession and its consequences: A pervasive reliance on quantitative metrics like Journal Impact Factor (JIF), H-index, and journal quartiles has led to a “compulsion to do research” rather than a “pleasure”, often resulting in “shoddy science” and high retraction rates. This obsession distorts research quality and has created systems prone to manipulation and “gimmicks”.
The flaws of current assessment systems: India’s primary assessment frameworks, such as the National Institution Ranking Framework (NIRF), heavily rely on publications indexed only in commercial databases like Scopus and Web of Science, overlooking valuable contributions from Indian journals and leading to “strange” and often misleading institutional rankings.
The role of academics and administrators: While external agencies and governments bear responsibility, academics themselves often perpetuate the misuse of metrics by succumbing to them in assessment committees due to what was described as “laziness” or a lack of ethical resolve. Administrators, too, find quantitative parameters “much easier” to use than qualitative judgment.
India’s unique challenges: India faces a complex situation characterized by a wide diversity of institutions and researchers, ranging from excellent to those with “very, very poor facilities”. Unlike some nations where problematic publications are a consequence of a huge volume of high-quality output, India has a smaller number of high-quality papers relative to a large quantity of problematic ones, making the issue more acute.
Pathways to responsible assessment: The panel underscored the need to move beyond journal-based metrics, value a broader range of research outputs, and implement explicit, transparent criteria for hiring, promotion, and tenure decisions. Practical suggestions included:
- Focusing on quality over quantity: Asking for a limited number of “key papers” (e.g., 5-10) and evaluating their quality through expert review, rather than relying on total publication counts or citations.
- Institutional autonomy and tailored policies: Empowering institutions to develop and clearly document their own assessment criteria tailored to their specific missions (e.g., teaching, applied research) and fostering “bottom-up” changes.
- Training and broadening Assessor pools: Providing concrete instructions and training to assessment committees to ensure they avoid simple numerical metrics and broaden the pool of evaluators to encourage more effort and less reliance on a small, busy group.
- Increased funding and infrastructure: Advocating for greater investment in research funding and infrastructure development in India, acknowledging that the current allocation (0.6-0.7% of GDP) is “too too small”.
- Addressing structural inequalities: Designing RRA practices that directly address systemic inequalities within academia and ensure broader representation of researchers in the design of these practices.
- Achieving critical mass and alignment: Recognizing that a “first mover disadvantage” exists in adopting new assessment practices, and therefore, a “critical mass” of organizations moving forward on several fronts simultaneously is essential for systemic change and alignment across funding agencies and institutions.
Read on or watch back on this inspiring event.
Rebecca Lawrence introduced DORA and its core themes advocate for moving away from journal-based metrics as a surrogate for quality, recognizing the value of a broader range of research outputs, and being explicit about criteria used in hiring, promotion, and tenure decisions. With over 22,000 individual and 3,500 organizational signatories across 166 countries, DORA’s mission encompasses raising awareness, facilitating the implementation of new policies, catalyzing broader reform, and improving equity by addressing structural inequalities in academia. To achieve this, DORA provides various tools and frameworks, including Reformscape—an interactive database of RRA practices, and a recently launched “Practical Guide for Responsible Research Assessment for Research Performing Organizations,” designed to help organizations navigate the overwhelming task of reform.
The event then had Moumita Koley setting the stage for the panel discussion, outlining the current research evaluation ecosystem in India. She detailed how research is assessed at various “touchpoints” – from PhD work and recruitment to external funding, scientific awards, and institutional rankings. Historically, India relied on peer judgment and peer review, but this has drastically shifted over the past 20 years towards a “publication and metrics-driven world”. Koley provided stark examples, such as the eligibility criteria for the prestigious Institute of Eminence scheme requiring institutions to rank within the top 50 in NIRF, with an expected outcome of featuring in the top 100 global university rankings like Times Higher Education (THE). She critically noted that NIRF’s research assessment is “entirely based on metrics,” including the number of publications (only those in Web of Science and Scopus), citations, journal quartile, and H-index. The consequences, she demonstrated, include a proliferation of “shoddy science” and high retraction rates in India, as documented in Science and Nature. She highlighted the absurdity of THE rankings placing institutions like Chitkara above the Indian Institute of Science (IISc) in research quality, despite IISc’s significant real-world contributions, emphasizing the disconnect between metrics and actual impact.
The distinguished panelists offered their unique perspectives on the state of research assessment in India.
Professor K. VijayRaghavan, DAE Homi Bhabha Chair Professor and former Principal Scientific Advisor to the Government of India, opened by stressing the need for a balanced perspective, acknowledging positive developments in Indian research despite a flawed assessment system. He contextualized the global shift in science from individual contributions to an institutionalized enterprise, leading to demands for accountability from taxpayers and the subsequent reliance on metrics. He highlighted India’s unique challenge: its diverse institutional landscape includes institutions of varying quality, making a “one-size-fits-all” approach ineffective. VijayRaghavan notably pointed out that “the biggest fault are our colleagues themselves,” who, when sitting on assessment committees, “succumb to using these metrics” due to “laziness” or a lack of ethical fortitude, calling for a change in this cultural problem.
Professor Subhash Lakhotia, Lifetime Distinguished Professor at Banaras Hindu University, echoed concerns about the “compulsion to do research” driven by quantitative requirements for career progression, often in settings with “very, very poor facilities”. He emphasized the “vicious circle” faced by Indian journals: low impact factors deter established scientists, preventing their growth, while administrators find quantitative parameters “much easier” to assess. Lakhotia advocated for a radical shift in bureaucratic procedures, suggesting that committees should ask for a limited number of key papers (e.g., five to ten) and evaluate them qualitatively, rather than relying on total publication counts or H-indices. He also stressed the need to balance basic and applied research, increase funding, and train assessors.
Dr. Michael Arentoft, Head of Unit of Open Science and Research Infrastructures at the European Commission, provided a European perspective, noting similar motivations for reform across the continent. He introduced the Coalition of Research Organizations for Advancing Research Assessment (CoARA), an initiative aimed at recognizing the diversity of research outputs and activities, emphasizing qualitative judgment, and explicitly prohibiting the use of JIF or journal-based metrics. Arentoft highlighted that CoARA has garnered over 800 signatories and fosters joint experimentation in concrete implementation of changes, with the Commission providing direct funding incentives and applying these principles within its own Horizon Europe program. He underlined the importance of “research on research” to ensure evidence-informed changes and stressed the need for transparency in underlying assessment data. Arentoft also emphasized the necessity for “coherence and alignment” between different assessment systems—for individuals, organizations, and research proposals—and the importance of training assessors.
Addressing Challenges and Moving Forward
The discussion also delved into concrete steps. VijayRaghavan emphasized that institutions must clearly articulate and document their specific assessment factors for hiring, tenure, and promotions, tailored to their missions (e.g., teaching or applied work). He underscored that these changes are “doable bottom up”. While Lakhotia lamented that he had “yet to see a good example honestly speaking in any institution” in India due to rigid university regulations set by the University Grants Commission, VijayRaghavan cited a few examples from well-funded research institutions like the International Center for Theoretical Sciences (TIFR) and, with some bias, the National Center for Biological Sciences, as well as historical examples from universities like BHU and new IISERs.
The panel reaffirmed that a critical mass of organizations implementing reforms simultaneously is vital to overcome the “first mover disadvantage”. Rebecca Lawrence concluded by announcing DORA’s plan to synthesize the discussion’s insights into a white paper, which will be circulated among major Indian funding agencies, institutions, and researchers to advance the RRA agenda.
This discussion underscored the urgent need for a paradigm shift in how research is valued and assessed in India. By learning from international experiences and fostering a collective commitment to responsible practices, the Indian research system can move towards a future where quality, integrity, and real-world impact are truly recognized and rewarded.
Practical Guide for Research Performing Organizations (DORA): https://sfdora.org/resource/practical-guide/
Professor Lakhotia’s Papers:
“Why we Publish, Where we publish and What we Publish?” (2014) DOI: 10.16943/ptinsa/2014/v80i3/8
“Why are Indian research journals not making a mark? – The enemy is within” (2018) https://www.jstor.org/stable/26978576
Coalition for Advancing Research Assessment (CoARA): http://coara.eu/
RESEARCH AI SUMMIT 2025: BRIDGING INNOVATION AND OPEN SCIENCE (New Delhi, June 9-10, 2025)
Indian National Science Academy Policy Statement on “Dissemination and Evaluation of Research Output in India”: https://www.nature.com/articles/nindia.2018.69 or https://insajournal.in/insaojs/index.php/proceedings/article/view/544
Speech by Prof. P. Balaram on “Greed, Vanity, and decline of scholarship”: https://youtu.be/ndtYh_rp7yw?si=VBNxWQufBkMPbclI
Upcoming Publications by Sangeeta Bagga-Gupta (connecting to the seminar):
Bagga-Gupta, S. (end-2025). Glocal entanglements, epistemic (in)justice and (dis)connections. Reflections on academic-life and -work on the contemporary planet. In Vincenzo Cicchelli & Isabelle Léglise (Eds.). The Craft of the Social Scientist in the Global Arena, Leiden: Brill.
Bagga-Gupta, S. (July-2025). Imaginings and re-imaginings in doing multiversal science. Editors’ introduction. In Bagga-Gupta, S. Ed. PaDELS, The Palgrave Handbook of Decolonizing the Educational and Language Sciences. Chapter 1. London: Palgrave.
Prof. K. VijayRaghavan
Professor Krishnaswamy VijayRaghavan is the DAE Homi Bhabha Chair Professor and former director of the National Centre for Biological Sciences, Tata Institute of Fundamental Research. From April 2018- April 2022, he served as the Principal Scientific Advisor to the Government of India. Prior to that, from 2013- 2018, he was Secretary to the Department of Biotechnology, Government of India. Till recently, VijayRaghavan was a Senior Editor at eLife and now is a member of its Board of Directors.
VijayRaghavan’s research interests are in the fields of developmental biology, genetics and neurogenetics, on the principles and mechanisms that control the nervous system and muscles during development, and how these neuromuscular systems direct specific locomotor behaviours.
In addition to continuing his laboratory’s research interests, VijayRaghavan is currently involved in studying how sustainable development can be enabled in the context of climate change and the energy crisis. In this context, his focus is on how the university system in India can be empowered to this task, and how industry collaboration in research can be enhanced.
VijayRaghavan is a Fellow of the Indian Science Academies. In 2012, he was elected a fellow of The Royal Society and was conferred the Padma Shri in 2013 by the Government of India. He is a foreign member of the US National Academy of Sciences, of the European Molecular Biology Organisation, and a member of the American Philosophical Society.
Prof. Subhash Chandra Lakhotia
Professor Subhash. C. Lakhotia, currently a lifetime Distinguished Professor at the Department of Zoology of Banaras Hindu University, obtained his PhD from University of Calcutta in 1970 for work on dosage compensation in Drosophila. He has been actively involved in teaching and research since 1971. His research has largely used the Drosophila model to study diverse areas like developmental gene expression and regulation, replication in chromosomes, cell stress response, long non-coding RNAs, neurodegeneration, tumor biology, Ayurvedic biology etc.
He has been proactively involved in discussions on quality of higher education and research, publication ethics, and assessment policies. Leading by example, Prof. Lakhotia is a strong votary of improving the quality of research journals published in India.
He was Editor-in-Chief of the Proceedings of the Indian National Science Academy and had been member of the editorial boards of Current Science, Journal of Biosciences, Cell Stress & Chaperones. RNA Biology and Annals of Neurosciences. An elected fellow of all the three science academies in India, and a Senior Fellow of the Cell Stress Society International (USA), he is recipient of several awards like INSA Young Scientist Medal, SS Bhatnagar Prize, UGC Career award, UGC National Lecturer, UGC JC Bose Medal, INSA Aryabhata Medal, SERB Distinguished Fellowship etc.
Dr Michael Arentoft
Dr Michael Arentoft is Head of Unit, Open Science and Research Infrastructures, DG Research & Innovation, European Commission. He was previously Deputy Head of International R&I Cooperation Strategy, Innovation Union policy officer, Acting Head of Strategy for ICT R&I, Sector Head of ICT R&I Work Programme and Planning, coordinator of ICT Essential Technologies and Infrastructures, and project officer in High Performance Computing and Networking. Before joining the EC, he was with Computer Resources International and with Rovsing International. His educational background is from the University of Pennsylvania’s Computer and Information Science PhD program and from the Technical University of Denmark’s Electrical Engineering Master’s program.
Dr Moumita Koley
Moumita Koley is a senior research analyst at the DST-Centre for Policy Research, IISc, Bangalore, India, and a Research Fellow at the Research on Research Institute (RoRI), UK. She worked as a consultant for the International Science Council’s Future of Scientific Publishing project between March 2023 and June 2024. Her work focuses on advancing the accessibility of scientific knowledge, fostering an efficient and responsible research ecosystem, and promoting research that responds to local needs. She is a member of the International University Association (IAU) Open Science Expert Group and serves on the steering committee of the San Francisco Declaration on Research Assessment (DORA), representing Asia.
Dr Rebecca Lawrence
Rebecca Lawrence is Managing Director of the open research publisher, F1000, now part of Taylor & Francis. She was responsible for the launch of F1000Research in 2013 and has subsequently led the initiative behind many funder-based publishing platforms for the EC, Gates Foundation, Wellcome and others, that aim to provide a new trajectory in the way scientific findings and data are shared.
She is currently Vice-Chair of DORA and was a member of the EC’s Open Science Policy Platform, chairing their work on next-generation indicators and Editor of their final report. She was a member of the US National Academies (NASEM) Committee on Advanced and Automated Workflows, and has been co-Chair of many working groups including for Research Data Alliance (RDA) and ORCID. She has worked in STM publishing for over 25 years and holds a PhD in Pharmacology.