Where did the inspiration for Peerage of Science come from?
‘Inspiration’ sounds something sudden, and personal. And that’s how it felt. It was beyond midnight, I was in bed, and there in the darkness it was all suddenly so clear. The three concepts that better peer review needs and how they tie to each other.
I had to get up and start writing it up. I wrote all night, and the basic outline of Peerage of Science was ready for colleague comments by 6 am. Those colleagues became my co-founders later on.
But of course that is not how ideas come to be, in reality. Ideas are never products of a single mind. Ideas do not spontaneously appear. They germinate unnoticed, in silence, in countless interactions with smart people, in happy events and, in particular, in disappointments.
On the day before, our research group had a journal club meeting, which had a tradition of spiralling discussions into philosophy of science, sociology of science and ethics of science, regardless of the original topic.
It was my turn to introduce something, and I had chosen a paper on peer review delay. Predictably, all the woes of peer review were soon on the table, not just delay. Amidst the furore, a comment “Someone should start an online service to fix this” was left echoing in my mind. Turns out communal ranting can be profoundly inspirational.
I recall the questions I had. How do I know I did not fail? How do I know if I did well? Did my peer review matter?
Four years before that point, I had started to receive reviewing requests from journals. I recall how good the first ones felt, like being invited to be a member of the scientific community. I recall the weight of responsibility. I recall the questions I had. How do I know I did not fail? How do I know if I did well? Did my peer review matter?
Over the years I learned that those questions are almost never answered. And I started to suspect that not very many people cared, either. Certainly the quality of my peer reviewing did not matter for my career as a scientist – but it mattered to me.
How does the process work?
The concept is to have a community-based process, where the technology is there to help the human process to self-organize and to facilitate human interactions productively. Peerage of Science is free for scientists, supported by payments from partnering publishers.
The Peerage of Science peer review process has four automated stages.
Stage 1: Author uploads the manuscript and sets the deadlines and anonymity rules. Peers can freely engage to review, and they complete structured, max 1000-word Essays by the deadline.
Stage 2: Peers judge and score each other on the quality of Essays. Importantly, they also leave written feedback on where they agree or disagree with the other reviewer.
Stage 3: Authors upload a revised manuscript, again by the deadline that they themselves set. If they can’t, they can choose to suspend the process.
Stage 4: Peers judge and score the revised manuscript – does it address the relevant issues raised in peer review? Recall that at this point they also know and have judged the comments made by other reviewers.
Authors or editors can both initiate correspondence. Editors decide if they want to invite a formal submission, and state the terms of that invitation (sometimes the Essays from Peerage are enough alone, sometimes editors state that they would seek one or more additional reviews in-house).
Authors may receive several invitations, and they decide whether to accept or decline. Or authors may opt to export the process to outside journal by creating a disposable account that has access to this one manuscript, and give that to editor of any journal.
What are the aims?
The most important aim is to measure, encourage, incentivize and reward high quality in peer reviewing work.
The most important aim is to measure, encourage, incentivize and reward high quality in peer reviewing work. And concomitantly, getting the community (scientists and journals) to admit the existence of the other side of that coin: sometimes peer reviews are really bad, and those failures need to be treated accordingly, not given the same weight as carefully written reviews. There should be consequences for excellence, and consequences for negligence.
Peerage of Science is not about doing peer review quickly, or with less effort. On the contrary, I want people to put more effort and time into doing careful, time-consuming, excellent peer reviews – but the idea is that you can do fewer of them, and still contribute more to science than you did before, when your reviews are excellent, and not discarded. A large number of journals can utilize the same set of reviews.
Of course, from an author’s and editor’s point of view, the whole publishing journey in Peerage of Science can end up being much, much quicker and more efficient than the old way. This is because instead of the author starting over repeatedly during a slow slide down the journal prestige ladder, the article-journal pairing can happen in one go. And the editor gets already-peer-reviewed submissions and can focus on making decisions in stewardship of his/her journal, while from author’s perspective the journal’s turnaround time is just hours.
How has Peerage of Science developed since it launched?
Over the four years there has been steady expansion of both scope and volume.
Over the four years there has been steady expansion of both scope and volume. But growing our volume – convincing more authors they should try breaking traditions with their hard-worked manuscripts – has been more challenging than we expected. Changing established behavioral patterns among scientists takes patience and determination.
Publishers and journals, on the other hand, have been more ready to give new ideas a try. Some visionary editors in ecology and ethology signed up their journals for a trial before we even launched. Then Pensoft and BioMed Central were our first publisher customers, first signing up journals in ecology, evolution, conservation biology and zoology.
Later we have established partnerships with many publishers and societies, largest among those our collaboration with Springer Nature, which brought in over 3000 journal destinations for authors through the Springer Transfer Desk.
I am very happy that this spring both of our first customers decided to expand their partnership with us, as Pensoft brought in the rest of their entire portfolio, and BioMed Central signed up four additional journals, BMC Genetics, BMC Genomics, BMC Plant Biology and the new journal BMC Zoology. It shows that as a service Peerage of Science has been doing something right, and is providing value for the community.
What’s next for Peerage of Science?
This summer and fall we are focusing on raising awareness among scientists – we often hear colleagues say “what a great idea, how come I have not heard about it before?”
This summer and fall we are focusing on raising awareness among scientists – we often hear colleagues say “what a great idea, how come I have not heard about it before?” For example, we will have booths at ISBE 2016 in Exeter UK, and at EcoSummit 2016 in Montpellier France – come and say “Hi” to us founders if you are there!
And a special day, August 25, is near too. Every year, Peerage of Science celebrates the anniversary day of the very first peer invitations, by awarding The Reviewer Prize to the peer with best reviewer performance over the preceding year.
We also expect to announce new publisher partnerships over the summer, bringing in quite a few more destinations for authors – stay tuned for that.
In the longer term, we would love to establish synergy partnerships with the other innovative young companies – there is a whole ecosystem of new things now in academic publishing, from collaborative authoring, through alternative metrics, to customized visibility and impact management and monitoring, and so forth. It would be great to find opportunities to make diverse solutions interoperable.
Comments