Luxembourg National Research Fund (FNR)

While the FNR has always fostered an approach to funding evaluation mainly based on peer review and qualitative aspects of assessment, to ensure that it funds the highest quality research possible, its approach has gradually evolved to embrace the principles and values of responsible research assessment (RRA). Specifically:

1.Research integrity

The FNR sees research integrity as central to research quality and aligns with international good practice to ensure that research funded by the FNR is at the highest standards of responsible research conduct. The provide a range of resources and initiatives for its applicants and grantees, including an open research integrity resource hub, providing a host of materials and guidance, and training in good scientific practice for PhD candidates. FNR also helped create the Luxembourg Agency for Research Integrity (LARI) in 2018, a national office that supports all institutions in good scientific practice, training, and investigation of alleged research misconduct.

As a result of these initiatives FNR manages to minimize the number research integrity cases among their funded research portfolio.

2.Updating FNR guidelines and processes with RRA principles

The FNR’s funding assessments are based upon qualitative approaches (e.g. peer review and panel discussions) which it has reinforced since becoming a signatory of the DORA declaration in 2019 and the CoARA agreement in 2022.

FNR has adapted its guidelines and processes to minimize the use of improper metrics in applications and evaluations (e.g. H-index and Journal Impact Factors), with clear explanations about the value of a diversity of research and career outputs included both applicant and reviewer guidelines. Reviewer panels are briefed on RRA, and FNR staff are required to ensure these procedures are followed during the funding decision-making process.

These measures have helped to significantly reduce the inappropriate use of metrics in research quality evaluation, and further facilitate a shift toward more qualitative and balanced approaches to research assessment.

3.Briefing reviewers and panel members on RRA principles 

FNR identified a need for guidance around the principles and practices of RRA for its grant reviewers and panel members. To address this, in 2021, FNR and DORA co-produced a video and a 1-page infographic.  The video provides six practical suggestions, including: addressing biases, ensuring group accountability, considering a broad range of contributions, balancing qualitative and quantitative information, and managing conflicts of interest. It is shown at every FNR panel and is also now used by funders worldwide.

4. Implementation of narrative CVs across all programmes

Previously, FNR proposals had no template or page limit in the CVs requested from applicants, leading to critical feedback from our evaluators due to the large effort (and low value) of the high documentation submitted. In 2021, the FNR introduced a narrative-style CV template (together with mandatory ORCID profiles) for Principal Investigators requesting funding from FNR programmes, to have a structured and simplified format while allowing applicants to demonstrate a broader range of contributions to science and society.

The template was based on existing initiatives (e.g. the Royal Society’s Resume for Researchers, as well as the templates from other funding agencies using narrative-style CVs) and supported by a communication and training package to both the national research ecosystem as well as our evaluators.

Following the implementation of narrative CVs, a feedback survey and analysis showed positive acceptance of the new CV format from both applicants and evaluators. The narrative CV is now normalized across the Luxembourg research landscape, and the learnings are used by others across the world in discussions and development of narrative CVs.

5. New bibliography to complement the narrative CV

In 2024, FNR complemented the narrative CV with a modified bibliography in project funding proposals, based on feedback from applicants and evaluators. The traditional bibliography was supplemented with a section for key outputs from the applicant consortium, along with context explaining their importance. The text from the application guidelines is below:

• Up to five most important outputs from the consortium that are relevant to the proposal. These outputs should include:
• The name of the output, with a link to it if possible (e.g. DOI or similar).
• A short description of the output, and why it is important and relevant to the proposed project.
• A short explanation of the consortium member’s contributions to the output – how were they involved in the process around the creation of the output.

6. Streamlined project reporting

Over the years, FNR has received feedback from national stakeholders that its project reporting requirements was burdensome and of limited value. In response, it developed an initiative to simplify the process for the research community, streamlining reporting across all grants to encourage a more reflective approach, reduce burden, and allow recognition of a broader range of research outputs.

The new approach aligns post-grant follow-up with the values of responsible research assessment used in pre-grant evaluation, asking consistent questions about what is funded and evaluated. Feedback has been positive, with reduced effort reported by researchers and administrative staff, though the shift to more reflective reporting remains challenging for both researchers and FNR staff.

7.Fostering diversity within and outside FNR grants

There are multiple national and European initiatives to improve EDI aspects in research, and the FNR believes that increasing equality, diversity, and inclusivity in their various processes will lead to higher quality research (and evaluations). The FNR has a group of policies and processes around EDI, centered around an Action Plan on Gender, Equality, Diversity, and Inclusion.

The Plan aligns with the requirements of the European Commission and outlines a set of measures and actions that promote gender equity and inclusion. This includes ensuring diversity in evaluation panels (with regards to gender, geography, and other aspects) and policies within funding schemes meant to improve gender balance in FNR-funded research.. The outcomes of these initiatives have been varied but positive: the evaluation committees are more diverse, and there is good diversity in the research the FNR funds. In addition, the Luxembourg research institutions are creating their own EDI initiatives, collaborating with each other and the FNR to achieve these goals.

8.Promoting and evaluating mentorship in research

FNR has introduced several initiatives to support and strengthen mentorship in research, including:

• In 2023, the Luxembourg research ecosystem published the Luxembourg Charter for Mentorship in Academia to underline the importance of mentorship in career development and in shaping the personal and professional lives of those in research. The Charter sets out values and competences for good mentorship, developed through discussions with the research community and mentorship award winners.
• Starting with the 2022 call of the PEARL talent attraction programme, former mentees of applicants are invited to evaluate the applicant’s mentorship abilities. As these grants support the development of sustainable research groups, mentorship is an important consideration.
• In addition, the FNR has a peer-reviewed award for Outstanding Mentorship in Research since 2020, supports international mentorship schemes (e.g. PEP-CV - a peer mentorship exchange around narrative CVs) as well as provides funding for coaching and mentoring within funded projects.

In 2019 the FNR signed the Declaration on Research Assessment (DORA) and in 2022 joined the Coalition on Advancing Research Assessment (CoARA), with the goal of aligning funding practices and processes to both initiatives. Furthermore, FNR's approaches align with the Global Research Council’s Dimensions of Responsible Research Assessment, CoARA’s commitments, and DORA’s recommendations, and FNR aim’s to continue to follow international good practice as practices shift around the world.

The FNR aims to support a diverse and inclusive research culture in Luxembourg by ensuring that research assessment is based on a fair and responsible process. The goals of the FNR are to fund the best research in Luxembourg, based on both what is being researched and how the research is being carried out. They believe that RRA enables a focus on the quality of what is funded while ensuring diversity, robustness, and openness of FNR-funded research.

As research is an international endeavor, with collaboration and mobility as core values, the FNR is keen to ensure that Luxembourg stays up to date with global trends, and the many institutions that are committing to RRA practices, through the work of DORA, CoARA and the GRC RRA Working Group for example. By keeping their practices and policies aligned with countries that have similar research quality standards, FNR helps to ensure that Luxembourg-based researchers of all levels are evaluated at the same standards of other high-quality research ecosystems. In this way, they can both collaborate with groups from other research ecosystems as well as be competitive when applying for grants or positions in other countries.

The FNR is committed to continuous reflection and improvement of its evaluation policies and processes, and thus constantly seek feedback from applicants, evaluators, and other stakeholders on how to improve how their funding and evaluation approaches. In addition, FNR staff regularly talk with other funders, engage in landscape analysis, and form internal working groups to understand what works and where improvements might be needed.

For changes to current practices to occur, buy-in from both staff and institutional leadership is essential. As a relatively small funding agency, FNR benefits from direct communication across the organisation and rapid timelines for discussion, feedback, and implementation. For larger initiatives (e.g. reporting guidelines, narrative CVs), small and agile internal project groups are created to develop and implement changes to evaluation processes.

It is important for any change to align with the values of the organisation, which enables the justification of change to colleagues, leadership and decision bodies, as well as the research ecosystem. The FNR’s values include ensuring alignment with international good practice and ensuring the highest quality of research evaluation to ensure that the best research is being funded.

Finally, any change must be clearly communicated internally and externally. FNR aims to stay closely connected to the Luxembourg research ecosystem and communicates changes at multiple levels, including yearly webinars for the national research community, targeted workshops and information sessions for researchers and administrative staff, clear and transparent website guidelines, and responsiveness to individual queries. Feedback on these efforts helps to improve guidance, and strong communication supports smoother change management for all involved.

The multiple initiatives described in this case study have spanned several years, which also indicates the FNR’s ongoing commitment to the continuous review and revision of its policies and processes.

Some of the initiatives built naturally on previous work, based on community feedback, internal reflection, and/or discussion. For example, the implementation of the narrative CV occurred in 2021, and the resultant feedback led to:

• Iterations in the template to remove issues that confused applicants (e.g. removing word counts and highlighting that non-mandatory sections).
• Production of the guidance video for evaluators to support their assessment of narrative CVs (and other RRA-based changes).
• Updating the bibliography to allow applicants to highlight their most pertinent accomplishments for proposals and their specific contributions.

In addition, the FNR is keen to ensure the acceptance of changes to processes among their research community through engagement, involvement and benchmarking. Building in feedback and adjustment opportunities processes increases the time needed to implement changes but are considered necessary to provide the evidence of benefits.