By Erika Shugart (American Society for Cell Biology)
Scientific societies have a vested interest in research assessment as standard bearers for their profession. We represent members who are at all career stages and in many different career paths. Societies have multiple roles in the assessment infrastructure. They publish scientific journals; they host large and small meetings; they provide professional development training; they give recognition through awards and fellowships; and they set standards for the profession. Collectively, this gives societies a variety of leverage points to affect change. At the same time, many societies are facing challenges ranging from changes in scholarly publishing, such as Plan S, to the American public’s increased disengagement with traditional organizations. As CEO of the American Society for Cell Biology (ASCB), I encourage scientific societies to explore how research assessment reform fits in their portfolio and can be a member benefit.
DORA originated at the 2012 ASCB meeting by a small group of stakeholders, including a number of ASCB members. ASCB staff managed the database of DORA signers for several years before partners joined to help us grow DORA into the initiative that it is today. ASCB’s early involvement with DORA complements the society’s work on equity. We have long-standing committees focused on women in cell biology, and minority affairs. We also have a newer committee composed of and focused on the career needs of graduate students and postdocs.
As a society, we can help our members individually and through changing the system. Cell biology, as part of the broader biomedical sciences, has had a hypercompetitive job market for academic positions for at least a decade. At the individual level, we provide professional development opportunities for researchers to share their science through meetings and publications, networking, and individual recognition through honorific awards and fellowships. A number of these activities involve assessment of an individual’s research. For example in the awards area, societies have the opportunity to consider the criteria they are using to assess nominees and the instructions that are given to selection committees. Awards are often given for accolades such as far-reaching contributions or exceptional scientific contributions, which do not have standard definitions. By working with award committees to come to a common understanding of these terms, we can help to ensure that the awards are fairer, which will raise the prestige of the award even more.
At a business level, most societies run scholarly publications. It is a simple matter to avoid misuse of the journal impact factor when advertising our journals. Instead we need to proclaim the advantages we have of being a member-focused publication and educate our authors and readers that “excess” revenue is invested back into the scientific community rather than lining the pockets of shareholders.
At a systems level, societies do a variety of work that is designed to strengthen the profession, which is an intangible benefit for members. For example, we advocate the importance of science to policy makers, and we collaborate on programs to improve inclusiveness in our respective fields. For example over 110 U.S. societies have banded together in the Societies Consortium on Sexual Harassment in STEMM, which is aimed to advance professional and ethical conduct, climate and culture in societies’ own operations, and STEMM fields broadly, in support of the inclusion of all talent and excellence in the fields. Research assessment reform can be part of this suite of initiatives. While societies can certainly work on the individual and business-level efforts that were highlighted above, our members will benefit the most if we change the system. Because societies are member-driven, we are uniquely positioned to do so.
Societies can take a number of actions to effect change:
- Sign DORA. The simplest step a society can take is to become a DORA signer, which indicates a shared value to their membership. Of the over 1,500 organizational signers of DORA less than 130 have the word society or association in their name. There are over 1,000 scientific, technological, engineering, mathematical, and medical (STEMM) societies in the US alone so there is plenty of opportunity for other societies to join DORA.
- Share examples of good practice. The next step is to share best practices with members, whether it is sharing examples from the DORA web page or contributing a case study to share with the global DORA community. DORA makes it easy.
- Engage with members. At ASCB we featured an interactive session ‘How to improve research assessment for hiring and funding decisions’ at our 2018 annual meeting, where participants provided feedback on grant and job application materials. The session was well attended and reviewed by participants. Another way that ASCB engages our members on research assessment reform is by promoting DORA’s community interview series.
- Examine our own practices through the lens of the DORA recommendations. At ASCB a lot of our efforts have been in publishing. Now we are considering experiments in poster selection and in the awards area. Most important, the ASCB does not use journal impact factors or other journal-based metrics in the evaluation of candidates for society awards.
Societies’ memberships span multiple institutions and even countries. If we are speaking with the same voice then societies can have a broad impact. By exploring ways to partner with other institutions and organizations that have best practices to share, we can play a key role in dissemination. I encourage all societies and associations to sign onto DORA and get involved.