As a panelist, I was asked to share what I see as the main challenges of open peer review. Having worked closely with the BMC series medical journals, which have operated on an open peer review model successfully since their launch (over 15 years ago), at first it felt hard to think of challenges.
To me, I see open peer review as an established model of peer review, with strong ethical arguments. I know from experience (both on the BMC series and as a co-Editor-in-Chief of Research Integrity and Peer Review) that it is feasible for journals to operate on this model. Despite this, it is not a universal method of peer review across different fields and disciplines.
This led me to my first challenge – the varying acceptance of open peer review by different research communities.
Attitudes towards open peer review
Despite well documented flaws of traditional single blind peer review, there is still scepticism about open peer review from many researchers.
When surveying preferences for different peer review models, and ways to reduce bias in peer review, time and time again double-blind peer review comes out as most popular among researchers (see examples here and here). This is despite evidence of the practical difficulties in blinding peer reviewers to authors’ identities.
There are fears that if a journal moves to open peer review and reviewers are required to sign their reports no-one will agree to peer review. Others worry that a reviewer won’t be able to provide a negative report on an open model.
What is the evidence?
Anecdotally these concerns are unfounded, but what is the evidence? Two randomized controlled trials conducted at the BMJ showed that the two aspects of open peer review (authors being aware of reviewers’ identities, and publishing reports) are both feasible. We also conducted some research that shows that open peer review may even be associated with better quality reports.
But this led me to my second challenge – the lack of research. There’s a lack of research, not only into open peer review, but into peer review in general.
What do we mean by ‘open peer review’?
A third, perhaps more pressing challenge, became clear to me as discussions went on. This is the challenge of agreeing what we mean when we say ‘open peer review’.
We need to be able to establish and define what we each mean when we say ‘open peer review’
Within the room of delegates, ‘open peer review’ meant something different to everyone. We need to be able to establish and define what we each mean when we say ‘open peer review’, particularly if we are to carry out research and generate more evidence into the process.
I consider the ‘open’ in open peer review to refer to transparency of the reviewers’ identities. I see it as a peer review models whereby reviewers’ identities are known to authors and, if the article is accepted for publication, their named reports are published alongside it.
This is just one definition of open peer review. Others see it as an umbrella term used to refer to a group of practices involving different levels and aspects of openness and transparency. This can involve post-publication peer review (both editor- and author- led), post-publication commenting and methods of involving the community in collaborative peer reviewing.
Some view the ‘open’ to also refer to peer review being ‘open’ to everyone – i.e. everyone who wants to can provide a review.
Where do we go from here?
This discussion shows me that, despite the feasibility of open peer review, there are challenges that we need to overcome.
Many of these challenges are interlinked. In order to increase the acceptance of open peer review more widely, we need more evidence that it works. Before such work is carried out though, we first need to define what we mean in order for that research to be valid and applicable.
It is clear to me that more research into peer review is needed in general. We are currently in a time of great innovation and as more and more versions and refinements of the peer review process arise, we need to establish what works for different communities. I believe that it is likely that different fields will need different approaches.
As well as research to assess whether different models of peer review lead to better quality peer reviewer reports, we more importantly need research to assess whether they lead to better quality articles – we need to not lose sight of the fact that that is after all what the process is trying to achieve.
.jpg)
Comments