By David Carr, Programme Manager – Open Research, the Wellcome Trust
In order to fully realise the benefits of open research, we must fundamentally change the way research is assessed. Wellcome is seeking to implement the DORA principles in our own funding processes and support research institutions in changing their own assessment practices.
As a research funder dedicated to improving health for everyone by enabling great ideas to thrive, the Wellcome Trust is committed to ensuring that the outputs of the research can be accessed and used in ways that maximise the benefit to health and society. For more than two decades, Wellcome has been a passionate champion and advocate of open access to research publications and research data sharing, and in 2017 established a dedicated in-house Open Research team to spearhead this work.
Deeply-engrained practices for research assessment serve as one of the most fundamental barriers to the wider adoption of open research practices. These place an overwhelming emphasis on publications as the primary research output that is valued, and often use the venue in which researcher publishes their work is often used as a proxy to judge both the quality of individual publications and the researcher’s work overall. As a result, many researchers will be reluctant to move to publishing their work in newer open access journals and platforms if they feel they will be disadvantaged in comparison to their peers by doing so. They will also not be incentivised to commit the time and effort it requires to share data, software and other research outputs of wider value if they feel that they will receive very little additional credit and recognition for doing so.
Changing our practices as a funder
Wellcome has a long-standing commitment to drive change. Since 2005, our open access policy has affirmed that it is the intrinsic merit of the work, and not the venue of publication which should be considered in funding decisions. In 2013, we became one of the first funders to sign the San Francisco Declaration on Research Assessment (DORA), and have worked to ensure that the expert reviewers and committees who are involved in assessing funding applications are clear on our expectations.
The expectations are clearly set out in our induction guidance for funding committees, which makes clear that those reviewing funding applications should:
- focus on the content and quality of publications, rather than their number, venue of publication, or the impact factors of the journals in which they are published
- take into account and value a diverse range of research outputs.
- be sensitive to legitimate delays in publication & personal factors that may have affected an applicant’s outputs.
These messages are reinforced in regular briefings to our committees, and our staff will intervene where they feel these principles are not being followed. We have also recently updated our application forms, which previously asked applicants to list their most significant publications. We now encourage applicants to include a range of research outputs alongside publications (including pre-prints, datasets and software) and to provide narrative to describe the significance of the outputs they select. We also require applicants to detail their plans for managing their research outputs and the resources they require to do so as an integral part of grant applications.
In implementing changes to our processes as a funder, there is a continued need for us to ensure that we strike an appropriate balance and avoid placing additional burden on applicants and reviewers. Where we request additional information, it also needs to be clear why we are requesting this and how we expect reviewers to use and assess it. There are also varying views on the extent to which funders should attempt to prescribe to our reviewers the basis on which they determine the scientific quality and potential impact of an applicant or research proposal.
Whilst we hope that the changes we have made so far are making a positive contribution to changing behaviours, we are under no illusion that the battle is won and there is undoubtedly more we could do. Changing these deeply embedded attitudes and practices takes time, and given the existing burden on academic researchers and the lack of alternative indicators and proxys, it is perhaps understandable that reviewers sometimes fall back on using venue of publication as a shortcut.
At present, while there is a growing recognition and buy-in for the need to change, the view in our communities still very much persists that where a researcher has published is a critical factor in determining their success in attaining grant funding and advancing their career. Convincing them otherwise will require leadership and coordination among the ever-growing number of stakeholders who are signing up to DORA and are committed to driving change. Together with a consortium of partner funders and publishers, we are actively supporting the implementation of DORA through continued outreach and the collection of good practice case studies that will help organisations develop and refine their approaches based on what has worked elsewhere.
Supporting change in research institutions
As a funder, we recognise that we are only one part of the system and can only do so much on our own. To achieve the change we seek, we also need the universities and other research institutions which employ the researchers we fund to embrace the principles of DORA.
In our new Open Access policy, which will come into force in January 2021, Wellcome has introduced a requirement that organisations in receipt of our funding must publicly commit to abide by the core principles set out by DORA, which they can do through signing DORA or an equivalent statement (such as the Leiden Manifesto). We will ask our funded organisations to demonstrate how they are putting this requirement into practice as part of regular audits we undertake and are developing guidance on the types of actions we would look for in this regard.
In order to support organisations to put these principles into practice, we are encouraging them to use funding we provide through our Institutional Strategic Support Fund to support dedicated staff and other resources to coordinate activities to implement DORA across the organisation – including developing their policies and guidance and rolling out training, communication and and advocacy activities. Several universities have established internal working groups bringing together researchers and other staff to steer these activities – this is one example of good practice we’d we keen for others to consider, and which we hope this funding can help support.
Work currently being developed by DORA to map the ‘pipelines’ and processes through which institutions have sought to implement DORA in practice will form a hugely beneficial resource to help guide institutions who are embarking on this journey to plan their own approaches and draw on the experience of others.
Looking forward to the workshop!
The DORA-HHMI workshop in October will provide a key forum to bring research institutions and other stakeholders together to share their perspectives and experiences of implementing changes to research assessment practices. It will hopefully provide a platform to develop new ideas and partnerships aimed at driving meaningful change, and I’m very much looking for forward to taking part.