Peer Review Week 2020 takes place September 21-25, with this year’s featured theme — “Trust in Peer Review” — dedicated to highlighting the importance of building trust in research.
To mark the occasion, we spoke with Dr Mario Malički, Co-Editor-in-Chief of Research Integrity and Peer Review, about the peer review process, transparency and how it continues to evolve.
What is peer review and why is it important?
Peer review is a quality control mechanism for scholarly research. It is a process that involves independent experts evaluating research proposals or completed projects of researchers. The experts themselves are also often researchers, and have similar skills, competencies and education to those whose work or proposals they are evaluating – this is why they are regarded as peers.
Peer review most commonly occurs when research proposals or projects are written up and submitted to a scholarly journal. Editors of that journal then invite experts to evaluate and scrutinize everything the researchers claimed to have done or are planning to do. This process is important as it aims to both give (or deny) a seal of approval for the submitted article and helps improve our knowledge through communication and advice that passes between the researchers and their peers.
How did you become interested in peer review as a research topic?
What I said above could be considered a general principle or a romanticized version of the peer review process. In reality, the quality of the peer review process itself is often unknown, and the whole process is very often unavailable for scrutiny. Most submitted articles are evaluated by only two to three experts, and those experts often disagree. Furthermore, who the experts are is often not disclosed to the scientists or to the public. So, I became very interested in how this process works, what it is that the experts look for, as well a what they criticize or suggest to scientists.
I also believe a part of my interest emerged during my training as a researcher, as the scrutiny I received from my mentors, was higher than any I received from peer reviewer for my first dozen articles. And I was painfully aware that some of my friends and follow young scientists were not so lucky.
Can you tell us about your journal, Research Integrity and Peer Review?
…we provide a venue to researchers that share the same curiosity about the functioning of science that we ourselves have
Our journal RIPR (we often refer to it using its ominous sounding acronym ripper) is a journal that specializes in research on the peer review process, research reporting, and research integrity. We are a multidisciplinary journal, as these topics are fundamental for all scholarly disciplines. Research on the methods and functioning of science is today often referred to as meta-science or meta-research.
As a journal, we are still in our beginnings, and currently publish approximately 20 to 30 articles per year. But we are proud of our achievements as we provide a venue to researchers that share the same curiosity about the functioning of science that we ourselves have. We hope to see both our journal and the field grow in the upcoming years, and maybe even play a part in the many changes coming to scholarly publishing.
This year’s theme for Peer Review Week is ‘Trust’. How can we encourage trust in peer review?
…when we are able to see all the steps it took for research to get published, we will be able to more easily make the decision if we trust or agree with what others have said or done.
I think if there was a simple solution, we would’ve applied it to many domains of human existence. It takes a long time to build trust, and only a few bad apples to have it shaken or destroyed. Nevertheless, I am an optimist, and I believe transparency goes a long way.
I have recently proposed the MAnuscripT CHanges (MATCH) taxonomy, that I am now piloting and refining with fellow meta-researchers. I hope it will one day be used across journals and publishers to describe the changes that occurred during the peer review of submitted articles, as well as between different versions of preprints, and between preprints and their (journal) published versions or record. I believe that when we are able to see all the steps it took for research to get published, we will be able to more easily make the decision if we trust or agree with what others have said or done.
Can you explain the peer review process in Research Integrity and Peer Review and how the journal ensures transparency?
When a manuscript is submitted to our journal, our editorial team conducts basic checks for the completeness of the submission as well as plagiarism detection. After that, it is assigned to one of the Co-Editors-in-Chief (Joerg or myself) and we will go over its contents to decide if the paper meets our journal’s scope or it if needs any changes before we invite experts to evaluate it.
We operate an open peer review system and ask our reviewers to sign their reports. In my time as an editor, only one expert declined to review a paper due to this policy. We also provide an option to experts to send us confidential comments they do not wish to be included in their published report. As editors, we provide our comments to the authors, and I will often have more questions for the authors than some of the reviewers. When the article is published, we publish the peer review reports and authors’ rebuttals. Finally, our journal allows authors to utilize Springer Nature’s In Review services.
And finally, what are the current discussions surrounding peer review and how is it evolving?
I am happy to see a lot of initiatives around endorsing or reviewing preprints
If I can be allowed a bit of direct promotion of our journal, I would recommend readers check a paper published by the late John Tennant (and co-authored by Tony Ross-Hellauer) as it provides a very good overview of our understanding of peer review. Another good starting point for those interested in peer review is Publon’s Global State of Peer Review.
As for what the future holds, and how it is evolving, I am happy to see a lot of initiatives around endorsing or reviewing preprints (e.g. PreReview, Review Commons, Peerage of Science, Plaudit), evolution of Scite.ai, links in search enginges and reference manager software for (post)publication comments on PubPeer, research on the means to evaluate the quality of peer review reports, and increase in the number of journals embracing open peer review practices.