The history of peer review, and looking forward to preprints in biomedicine

The report from SpotOn, 'What might peer review look like in 2030?' has now been published. This blog contains a section on the history of peer review from Frank Norman. Read the full report from SpotOn 2016 here.

Peer review is not as old as you might think

Peer review is often regarded as a ‘touchstone of modern evaluation of scientific quality’ but it is only relatively recently that it has become widely adopted in scientific publishing. The journal Nature did not introduce a formal peer review system until 1967. Before then some papers were reviewed, others were not. Michael Nielsen suggests that with the ‘increasing specialization of science…editors gradually found it harder to make informed decisions about what was worth publishing’.

Aileen Fyfe has pointed out that ‘peer review should not be treated as a sacred cow … rather, it should be seen for what it is: the currently dominant practice in a long and varied history of reviewing practices’.

Challenging the status quo

The widespread adoption of the Internet as a means of scholarly interaction began in the mid to late 1990s. Even back then discussions raged about the benefits and disbenefits of challenging the publishing status quo. Tony Delamothe, writing in 1998, summed up the arguments thus:

At one extreme were enthusiasts for electronic preprints, who regard them not as scientific papers in evolution but as near enough finished articles. To these respondents, the current long process of peer review and paper publication is detrimental to science and the public health: any way of getting scientific advances into the public domain fast is worth supporting.

At the other extreme were respondents who thought “too much junk” was already being published. Lacking the skills to distinguish between “valuable material and garbage” journalists and the public could be misled.

More recently the realization has been growing that researchers will use electronic preprints because of their benefits—however much journals may rail against them.

The following year it seemed that the world was really changing when the US National Institutes of Health published its E-biomed proposal but this proved too radical for many in the biomedical research community.

Scientific reports in the E-biomed repository would be submitted through either of two mechanisms… (i) Many reports would be submitted to editorial boards. These boards could be identical to those that represent current print journals or they might be composed of members of scientific societies or other groups approved by the E-biomed Governing Board. (ii) Other reports would be posted immediately in the E-biomed repository, prior to any conventional peer review, after passing a simple screen for appropriateness.

That last part seemed too big a departure from peer review, and the proposal was watered down, leading to the establishment of the PubMedCentral repository for published papers. The proposal indirectly stimulated the creation of two new publishers – the commercial BioMedCentral and the not-for-profit PLOS.

The early impact of open access on peer review

Early proponents of open access took pains to make it clear that their immediate goal was to improve access to research literature, and not to challenge peer review practices. They were careful not to lose the support of those who held peer review dear.

However, as new open access journals were established they did provide opportunities to experiment with enhancements to peer review. PLOS ONE, launching in 2006, famously popularized the idea of a ‘megajournal’ with its mission to publish “scientifically rigorous research regardless of novelty”. This model was followed by a swathe of other megajournals. The Frontiers series of journals launched in 2007 and introduced ‘interactive collaborative review’, which aimed to turn the peer review process into a “direct online dialogue, enabling quick iterations and facilitating consensus”. In 2012 eLife launched with the aim of ‘taking the pain out of peer review’, again by a more collaborative approach. Gradually, the role of peer review was challenged and the practice changed. PLOS ONE introduced the idea that dissemination of research was at least as important as validation of research.

Happily, these challenges have not caused the whole world of research communication to come crashing down. In recent years there has been a bit of a rash of retractions, but these are more strongly associated with high end journals than with megajournals. The strong positions that PLOS ONE and Scientific Reports have
achieved suggest that megajournals are here to stay.

A few journals have sought to modify peer review further – F1000Research and Wellcome Open Research make preprints of articles available almost immediately after submission, and then invite post-publication open peer review.

If most research articles are posted first as preprints then access to research findings becomes possible as soon as an article is completed rather than, as at present, when the article is accepted and published in a journal.

Preprints

A preprint is “a scientific manuscript uploaded by authors to an open access, public server before formal peer review”. Currently proponents of preprints are following the same strategy as the early OA advocates. Preprints are advocated as a route to better access rather than as a challenge to peer review. The ASAPbio initiative is
all about faster access to research findings – ‘Accelerating Science and Publication in Biology’. If most research articles are posted first as preprints then access to research findings becomes possible as soon as an article is completed rather than, as at present, when the article is accepted and published in a journal.

Challenges to the current practice of peer review will surely follow the wider adoption of preprints.

Preprints have been widely adopted by physicists through the ArXiv server, but publishing practices and sharing cultures vary greatly between different research fields and biomedical researchers did not show much enthusiasm for preprints until recently. ArXiv has provided a home for computational biology preprints, and this
helped to pave the way for the establishment of bioRxiv – a preprint server for biomedical sciences.

Most articles uploaded to bioRxiv are also submitted to journals and subsequently peer-reviewed and published. But some also see bioRxiv as a permanent home for research results. One researcher has declared that one of his bioRxiv preprints is the “final version” and that he will not submit it for publication in a journal. Partly
this is because the article is a response to a previously published article, rather than a full article in its own right. But the researcher also wanted to experiment with how preprints are perceived by researchers.”

Preprints are still a tiny fraction of the total output of biomedical research papers. If there is widespread adoption, and researchers become accustomed to reading research reports that have not been peer-reviewed, we may increasingly question the value of peer review as a means of screening all research reports. Bernd Pulverer has suggested that:

“If preprints should attain the dominant role they have in physics, publishing papers in journals may remain attractive only in journals that add real value to the scientific communication process.”

He suggests that it will be worthwhile only for quality journals “to invest time and effort to add reliability and reproducibility assurances to research findings through careful peer review and prepublication quality control and curation processes.”

We may be moving to a world where some research is just published ‘as is’, and subject to post-publication peer review, while other research goes through a more rigorous form of review including reproducibility checks. This will be a gradual process, over a period of years. New tools such as Meta and Yewno, using artificial
intelligence, will help by providing new ways to discover and filter the literature. A new set of research behaviors will emerge around reading, interpreting and responding to preprint literature. The corridors of science will resound with caveat lector and nullius in verba.

 

Further reading

1. Baldwin, M. “In referees we trust?” Physics Today 70:2 (2017), 44, doi: https://dx.doi.org/10.1063/PT.3.3463.

2. “A succinct history of academic peer review.” Frontiers blog (2015). Retrieved Mar 10 2017 from: https://blog.frontiersin.org/2016/09/15/a-history-
of-academic-peer-review/.

3. Keroso, N. H. “Open and Post Peer Review: New Trends in Open Access Publications.” UA Magazine (2016). Retrieved Mar 10 2017 from: https://www.ua-magazine.com/open-post-peer-review-new-trends-open-access-publications/.WMLAhNLyi70#.WMaMGdKLS70

4. Flier, J. S. “It’s time to overhaul the secretive peer review process.” STAT (2016). Retrieved Mar 10 2017 from: https://www.statnews.com/2016/12/05/peer-review-process-science/comment-page-1/.

5. Patterson, M. “Bringing eLife to life.” Insights 26:3 (Nov 2013).

6. “Collaborative Peer Review.” Frontiers. Retrieved Mar 10 2017, from: https://home.frontiersin.org/about/review-system.

Share this post

View the latest posts on the Research in progress blog homepage

Comments