Evidence of publication bias favouring positive results, both in which studies are reported in journals and in how they are treated by peer reviewers, has emerged at the 6th Peer Review Congress in Vancouver.

Seth Leopold described how in his group’s study 2 variations of a fabricated, CONSORT-conforming randomized controlled trial (RCT) were sent to 209 peer reviewers for Journal of Bone and Joint Surgery and Clinical Orthopaedics and Related Research. Peer reviewers were randomized to receive each version of the paper, which were identical apart from the direction of the finding – one reported a positive result whereas the other “no difference”.

Testing for the Presence of Positive-Outcome Bias in Peer Review: A Randomized Controlled Trial
Gwendolyn B. Emerson, Richard A. Brand, James D. Heckman, Winston J. Warme, Fredric M. Wolf, and Seth S. Leopold

They found that the paper reporting a positive result received more positive reviews for methodological rigour, was scrutinized less carefully for errors, and was more likely to receive recommendations to publish.

Moreover, a further analysis of published vs. unpublished Food and Drug Administration (FDA) data on antidepressant trial outcomes, which now includes ‘published’ trials if they appear in review articles, found negative trial results were bundled with positive ones in reviews, rather than being fully published. Of 8 trials reviewed by the FDA in the study period, half of which were deemed positive and half negative, outcomes were reported positively 103 times compared to 8 times negatively.

Multiple Publication of Positive vs Negative Trial Results in Review Articles: Influence on Apparent Weight of the Evidence
Erick Turner

Two other abstracts presented on 11th September investigated the time to publication of manuscripts rejected by major biomedical journals and a common finding was that a quarter of manuscripts in the samples remain apparently unpublished, a number of years of later.

These findings – although not yet published in peer-reviewed journals – seem to add to the evidence of publication bias favouring positive results, and suggest unnecessary gaps in the literature and wasted research efforts continue.

But a number of BioMed Central journals actively encourage publication of these data. The journal, Trials will consider RCT results regardless of outcome or significance of findings, and BMC Research Notes aims to complete the scientific record by publishing scientifically sound research across all fields of biology and medicine. 

Last month Trials published 6 new research articles (as well as 8 other articles), including the study by Pibernik-Okanovic et al., which found that psycho-educational treatment for depressed diabetics was no better than the standard care provided to the control group.

The Peer Review Congress – held every 4 years – aims "to improve the quality and credibility of biomedical peer review and publication and to help advance the efficiency, effectiveness, and equitability of the dissemination of biomedical information throughout the world."