Are there alternatives to peer review?

There are lots of discussions about peer review happening at the moment, it seems April is a busy month. My colleagues took part in peer review debates in London and Doha discussing innovations, and on Friday I was at a Sense about Science workshop called ‘Peer Review: The Nuts & Bolts’.

Photo from @LaurenTedaldi at @voiceofyoungsci with permission
Photo from @LaurenTedaldi at @voiceofyoungsci with permission

The event was for early career researchers, exploring how peer review works, how to get involved, the challenges to the system, and the role of peer review in helping the public to evaluate research claims.

We heard the perspective of the editorial office from  Alice Ellingham, Director of Editorial Office Ltd, of the peer reviewer  from John Gilbert, Editor of the International Journal of Science Education, and of the Editor – me.

While the importance of peer review is generally appreciated (see this helpful guide written by Sense about Science on peer review) it’s difficult to point to real ‘hard evidence’ of its value as Cameron Neylon has previously commented.

There are a few studies that suggest peer review is somewhat better than throwing a dice and a bunch that say it is much the same.’ Cameron Neylon

Anecdotally, we agreed that peer review at its best can be informative, constructive and even forge new collaborations. However, participants at the workshop identified more negatives than positives with the current system. It’s slow, inefficient, political, prone to bias. Reviewer reports can be superficial (or worse completely over the top). Goalposts change, Editors embark on endless rounds of re-review, reviewers and editors can go AWOL.

Early career researchers are rarely involved in the process and there are few incentives to do peer review. However, some of the suggested solutions from the audience may be impractical, such as presenting findings to a panel, hosting dedicated peer review conferences or crowdsourcing peer review. It seems the present system is here to stay – so how can we improve it?

Photo from @easternblot at @F1000Research with permission.
Photo from @easternblot at @F1000Research with permission.

Discussions touched on some of the solutions (old and new) to improve things:

Open peer review. The medical journals in the BMC series operate open peer review, as do the biology journals Biology Direct and GigaScience. F1000Research and the British Medical Journal also value openness.

Avoiding re-review. BMC Biology offers its own solution to avoiding endless rounds of re-review, and the streamlined decision making process at eLife addresses this too.

Peer review of peer review. Reviewers at Peerage of Science feedback on each others’ reviews and Janne-Tuomas Seppänen’s motivation for starting this service is explained here.

Decoupling peer review. As with Peerage of Science, Axios Review and Rubriq are separating the review process from the journal.

Post-publication peer review. F1000Research  champion post-publication peer review.

Sharing peer reviews. A number of consortiums e.g. eLife consortium and Neuroscience Peer Review Consortium will allow the transfer of reviewer reports and identity (if the reviewers are willing) to other journals. Authors who are not accepted by their first choice journal, will be able to submit their manuscript (with reports) to any other journal in the consortium if they wish.

Collaborative peer review. eLife and Frontiers journals enable reviewers to discuss the manuscript among themselves before communicating a unified decision to the authors.

Crediting peer review. Open peer review goes some way to address the lack of credit for reviewers, however, F1000Research go further in making reviews fully citable with DOIs. They are doing exciting work with ORCID to linking review activities with ORCID identifiers. Also citable ‘reviewer acknowledgements’ give credit, even in situations where the original manuscript is rejected.

And of course, Mentoring peer review. There is a real need to include early career researchers in the process. Given they may not have the necessary publication record to be approached directly, PIs have a role to play in involving their students, with the Editors’ consent, in peer review. COPE has produced helpful ethical guidelines for peer reviewers which is great resource for coaching students.

It was good to learn that Sense about Science are behind a campaign to recognise reviewing in the Research Excellence Framework (REF). Certainly, there have been some inventive approaches to improving the peer review process since Martin Raff and colleagues, and Hidde Ploegh called for changes to the system back in 2008 and 2011 respectively.

It’s an exciting time to be in the business of peer review, thank you to Sense about Science and participants for an interesting morning of discussions!

Upcoming events:

  • Todd Hummell will speak during the session on Open Peer Review at Council of Science Editors meeting in San Antonio, 2-5 May.
  • BioMed Central’s Biology & Medical Editors are organizing a panel debate on peer review during an Editors’ conference in London, 14 May.

View the latest posts on the Research in progress blog homepage