Professor Jim Reekers has started to notice that more reviewers are sending conflicting decision recommendations to the editor. In this blog post he considers why this might be.

As Editor in Chief of an emerging open access journal, I have started to notice that some papers, sent out for peer review, tend to evoke very contradictory opinions. Reviewers sending conflicting recommendations to the editor: reject versus accept, leaving out minor or major revisions. Of course this can happen, but this is not to be expected too often from objective peer-review, and I was urged by this observation to do some research.

Dichotomous peer review

I found that some of these dichotomous peer review results were seen in papers that had a negative outcome for the investigated treatment. Moreover, all these controversial papers described a negative outcome regarding treatment options that had been found very positive, or at least promising in earlier, mostly industry-sponsored and heavily promoted, research papers. It looked more like rejecting new findings that conflicted with the existing beliefs of the reviewer. Also, most of the negative (reject) reviews were short and not dealing with the methodology or statistical analysis but concentrated mainly on the results. Some reviewers wrote quite frankly that they did not believe the results or that they found them not to be in line with what we already know, disregarding the presented science.

Cognitive dissonance

The only way science can make progress is by discussions, judging the pros and cons with scientific arguments. My observation indicates how powerful the promotional activities of the pharmacological companies are, and how they can implement a general cognitive dissonance among their customers regarding their products. This dominance of industry-driven science and the adherent promotional campaigns are therefore a serious threat to the scientific truth. This will impact not only the quality of the science, but also has a direct influence our evidence-based treatment guideline. As an editor in chief, I would like to call upon my fellow EICs, in case of studies which do not confirm the general accepted state of the art science, to give these papers more often the benefit of the doubt. Publishing the well-known common opinion does not add to the scientific discourse.

Blocking new science

Journals with a high impact factor receive an overload of submissions, and the consequence of this is that the rejection rate of these journals is very high, sometimes >80%. For an editor in chief of such a journal, a high percentage of rejections by peer-reviewers is a blessing from heaven. However, based on the cognitive dissonance in peer review, we have to assume that probably some of the newest, most innovative or challenging papers are part of this bulk of rejections. If this is true, it means that progress in science is blocked and that the mainstream scientific opinion will not be challenged. Publishing science on preprint sites, bypassing peer review, is however not a solution as science without scrutinization by peers leads to a parallel scientific world full of fake science and conspiracy.

Dedicated open-access journals

To overcome this problem, we have to take some urgent steps. The first is to increase the number of smaller, dedicated and more specialized open access journals, with peer-review as a quality gatekeeper, but doing so to increase the number of (subspecializing) publications. Secondly, we have to abandon the false perception that an impact factor is a parameter for the quality of a journal. The impact factor is only a measurement of the perceived reputation of a journal. Moreover, the IF is also not a quality parameter for an individual paper. The citation score and the Altmetric score for an individual paper would be much better and more realistic, reducing the current dominant role of bibliometric indices. Finally, open access journals will be a powerful promotor for broad dissemination of new scientific ideas, and this will therefore bring the scientific discourse to a higher level. To stimulate and to improve scientific discussions and implementation of new ideas, governmental organizations should always include mandatory open access publication in every grant provision.