GigaScience’s Peer Review Process Goes Meta via Blogs and Twitter

Posted on behalf of Laurie Goodman

Seeing a blog expounding about a manuscript that is currently under peer-review — by one of the reviewers— is a nightmare come true for most editors – not to mention the authors. The terms of strict confidentiality about submitted manuscripts are quite clearly stated in the Guide to Reviewers, which effectively says no outside discussion. This was the exact situation that we at GigaScience found ourselves in when we discovered that Titus Brown, who was reviewing the then submitted Assemblathon 2 manuscript, had posted a blog on it the day after submitting his review.

Our moment of shock and “What now?!”, however, was quickly followed by a surge of “This is fantastic!!” as we realized that firstly there was no breach because the authors had posted their paper in a pre-print server — something we at GigaScience encourage because our goal is to increase the speed of scientific communication. Thus the authors themselves had already “broken” confidentiality —or perhaps we should say they had “repaired” the communication process. Secondly what we were seeing was a rather unique peer review process. One that was, well— better.

Here’s how it went: First, we had reviewer Titus Brown’s blog providing his thoughts about the manuscript after his first round of reviewing it— but instead of giving a series of “this was wrong” and “that was wrong”, or comments focused on “the authors really need to do X before this should ever be published”, it was a thought-provoking discourse about what he felt the manuscript had revealed about the status of genome assembly work in general, both from a technical and a cultural side. It added something to the peer review process— a positive something. It moved it to a more forward-thinking process, rather than a “what we know now and 5 minutes ago, and why it’s not good” process.

Then… there appeared a new blog from Homologus about —Titus’ blog! This wasn’t a rehashing of what Titus wrote. Instead it was another step forward focusing on broader issues that the community needed to think about concerning the state of computational biology in general. We saw more blogs following these such as a post from In between lines of code.

Next… the comments leapt to Twitter, and these included tweets from the authors encouraging people to contact them if they wanted to give input into the revisions they were in the process of making. Thus, they clearly were happy with the open discussion, and were themselves encouraging it.

This was the point when I realized that this was the first peer review in my 20+ years of publishing that was— well— fun. And, best, it was productive. Given this, we are considering ways to promote this process at the journal— where the level of confidentiality is solely the author’s decision. As a start, we are modifying our Guide to Reviewers confidentiality statement from “Any manuscript sent for peer review is a confidential document and should remain so until it is formally published” to now include: “Exceptions: As we are promoting and encouraging more transparent review and the use of pre-print servers, if the authors and reviewers consent then we do allow open discussion of the work prior to publication.” Note: when there is a ‘crowd-sourced’ review, our decisions will be based on the comments from our chosen reviewers, not on the ongoing comments. But we feel that having reviewers and the community having the option to weigh on a paper pre-publication in an open and productive manner is definitely something to be encouraged.

For other ways GigaScience is promoting transparent peer-review see the previously published paper on SOAPdenovo 2 and the accompanying Editorial describing open-peer review and full data and tool review.