BioMed Central journals BMC Biology, BMC Ecology, BMC Evolutionary Biology and Frontiers in Zoology now welcome manuscripts that have been reviewed through the community peer-review initiative Peerage of Science.

Much has been made recently of the imperfections of the traditional peer-review process as a way to evaluate scientific knowledge. Despite quibbles about process, the current system still represents the gold standard which—to paraphrase Winston Churchill on democracy—might be said to be the worst form of scientific assessment, except all those other forms that have been tried from time to time.

A key problem for journals and journal editors is balancing efficiency, rigour and fairness. All three are crucial, and interwoven with each other to affect the scientist as author, editor, reviewer and reader.

To address this, a little under year ago a new initiative called Peerage of Science was launched aiming to tackle some of these underlying issues and to reward reviewers in the process.

Processes and positive

The basic idea behind the initiative lies in the creation of a “community of peers”, each signed up to submit and review manuscripts in a transparent format. Any scientist who has a publication record is free to join the community, and those without can only become members if their manuscript is favourably reviewed. Manuscripts are submitted anonymously to a central pool, and any peer can then choose to review it. Submitting a manuscript creates a “peer review debt” of -2 reviews, which is shared among all authors. Writing a review increases this personal balance by +1, with subsequent submissions only being possible if the balance is positive– the aim being to ensure reciprocal participation in the review process.

Authors specify the time limit for their manuscript to be reviewed, and once this expires all submitted reviews are collated. Each reviewer’s evaluations are also reciprocally reviewed, and each reviewer is given a score according to how the other perceives the quality of their assessment, termed the Peerage Essay Quality Score (PEQ-score). Editors of relevant journals are able to track the progress of manuscripts through the peer-review process, allowing them the opportunity to eventually  “bid” for them to be published in their journal – the aim being to avoid “bouncing” manuscripts around journals that may otherwise not find them suitable. Because manuscripts are evaluated only once, reviewer effort and review time is saved.

But it is not just authors and journals that stand to benefit from this process. Good reviewers are rewarded by accruing a “referee factor”, in the form of their accumulated PEQ score. Although this score is accrued anonymously, peers also have the option to openly publish their review in a citable form, allowing them to grow their publication list.

This system thereby encourages strict anonymity during the review process, and openness after it.

One of the added benefits of encouraging an open evaluation of peer-review is that it can act as a training platform for the next generation of peer-reviewers. The founders of Peerage of Science explained in a recent interview:

“[As a referee] when you start to write review reports, there is little feedback to tell you if you did well or failed.

In Peerage of Science you not only get to see a lot of reviews, but also see what others think of them, and you get clear, quantitative evaluation of your own performance, too. When some people start to gain reputation as excellent peer-reviewers, their Peerage Essays…are examples to students of what a high-quality peer review looks like.”

BioMed Central has always strongly supported innovations that increase the openness and efficiency of peer-review. All Medical journals published within the BMC-series of journals operate fully open peer-review, with publication of their pre-publication history for anyone to access. Biology Direct has its own system of open peer review, in which the author selects the reviewers; and BMC Biology operates a policy of re-review opt-out aimed at reducing frustrating iterations.

We welcome innovations that may lead to greater fairness and efficiency. An excellent overview of the modified form of peer-review operated by Peerage of Science can be found on their homepage, and in this independent overview (particularly Fig.1).

Caveats

Much like open peer-review, the ideas behind Peerage of Science have met with some understandable scepticism from the research community—something which its founders recognise as inevitable with any new venture that alters an established process.  The idea originated in the ecology and evolution departments of University of Jyväskylä and the University of Eastern Finland, and it is among these fields that support has largely grown– as would be expected from a system of invitation-only peerage.

One accusation that has lingered about this approach is: how will it avoid problems of “nepotistic reviewing”, as colleagues and friends review each other’s work?

Co-founding peer Janne-Tuomas Seppänen explains:

“In Peerage of Science peers can freely choose what to review. Editors of fully participating journals have tools to invite reviews for a given manuscript from particular peers, but the same tools are available to everyone else as well, including the authors.

But friendships or feuds are difficult to control for.

Chances are that in the traditional system a nepotistic (or negatively biased) review goes through undetected, as…there is usually no peer review of the reviewer’s arguments. In contrast, Peerage of Science features peer review of peer review, public reputation incentive, plus full audit trail of every action in the system and severe punishment for scientific misconduct.

[The]Database is screened regularly, so reciprocal nepotistic reviewing can be detected years after the offense, and the consequence is public expulsion from the Peerage of Science community, for life. So I firmly believe peers in Peerage of Science rather use their time to honestly build a strong reviewer reputation, than risk that reputation for small and uncertain cheating gains.”

Supported by BioMed Central

More than 1000 peers have now joined the Peerage of Science Community, to help kick-start this new venture – a number of which are currently Editorial Board Members of BioMed Central journals BMC Ecology, BMC Evolutionary Biology and Frontiers in Zoology.

Our support of this development in peer-review largely reflects the support afforded to it by the members of our Editorial Boards, who are intrinsic to the development of both our journals and the field as a whole.

Asked why he chose to sign up as a peer, Associate Editor for BMC Evolutionary Biology Lutz Fromhage explained:

“I got interested because both as an author and as an Associate Editor I have occasionally been frustrated with the traditional peer-review system. I saw this initiative as a chance to harness  the full potential of the online world to make peer review more efficient. On the other hand I was sceptical about some aspects, so joining seemed like the best way to form an informed opinion.

I believe that peer-review will have to change profoundly in the long run, whether through Peerage or through other systems that are yet to be invented. Staying closely in touch with such developments is probably a wise move with regard to the long-term success of our journal.”

Similarly Josef Settele, Section Editor for BMC Ecology, sees this as

“a wonderful opportunity to make the review process more transparent and to see what people write and intend to publish…because we can contribute to a scientifically sound and challenging procedure. I think it is always good if you show that you are deeply rooted in scientific processes.”

Evolution of an idea

Although only four BioMed Central journals have so far joined the list of supporting journals which welcome links, this partly reflects the fields in which this initiative has developed. We eagerly wait to see how other fields take to this new format, and hope to grow our support in unison.

We are also not currently utilising one of the many innovative features of the system – “bidding” for suitable manuscripts.  Principally this is due to policy. All journals published in the BMC-series (with the exception of BMC Biology and BMC Medicine) are broad in scope, and aim to publish work deemed by peer reviewers to be a coherent and sound addition to scientific knowledge and to put less emphasis on interest levels, provided that the research constitutes a useful contribution to the field.

Much like the ethos of Peerage of Science, our aim is to help researchers to get their sound science published, wherever possible. We are happy to consider manuscripts in all areas of ecology and evolutionary biology and would certainly welcome submissions from anyone working in these disciplines.*

Of course as research fields, publishing and publishers change, we are open to a revision of this policy.

The landscape of scientific publishing has changed dramatically over the last few years, and growth in the volume of scientific articles has also meant an increased pressure on researchers in specific fields to review articles. Partly this reflects growth in science, partly changing attitudes towards the accountability of review. Would Watson and Crick’s original paper on the structure of DNA have been accepted without review in today’s publishing climate? Similarly Einstein’s theory of special relativity? Almost certainly not, though this we imagine would be seen as a positive development by most in the scientific community.

We’re happy to be part of a system that tries to streamline this process for everyone involved – without compromising on quality.

*Please note that Frontiers In Zoology does not share the same policy for consideration as the BMC-series