87% of the most highly cited articles are authored by fewer than 1% of scientists. That is the finding of a recent study by John Ioannidis and colleagues published in PLOS One.
The authors examined the entire SCOPUS database to determine the number and characteristics of scientists who had an uninterrupted continuous publication record (UCP) for a period of 16yrs from 1996 to 2011.
Out of over 15 million authors, 150,608 had such a UCP record, and of those, 73.3% could be deemed ‘successful’ as determined by citation metrics and the author’s H index – a measure of their productivity as well as the impact of their work. This proportion is significantly diminished to 43.7% for researchers who skipped only one year.
In analyzing the characteristics of the researchers the authors find that medical research, physics and chemistry are areas where this UCP pattern predominantly occurs and less so in computer science and the social sciences. The authors speculate on possible reasons why continuous yearly publications are necessary for success and how this might be more important for some fields rather than others.
They also discuss barriers to continuous publishing, such as poor research infrastructure and the unavailability of long term grants. It is not news that a minority of scientists should be so influential, but the size of the minority (1% !) is a surprise.
There is no doubt that this article raises many questions and avenues for further discussion. In particular, how far does this minority rely on the efforts of junior colleagues for their success? It also provides further evidence of the pressures scientists are under to publish if they wish to have successful career – a pressure that drives some to unethical conduct.
What is disappointing to me about this article is that the authors merely touch on gender related issues or, ‘child bearing’ as they call it, as possible reasons for the small minority in their discussion.
For some fields, including medical research, the implication is that a researcher would have to publish continuously for 16 years in order to have any chance of joining that elite minority, and even a break of one year diminishes the chances of success. This has grim implications for women researchers who wish to take a career break to have children.
The UK government not only recognizes the need for more scientists and engineers, but the economic value of women scientist. There have been drives to encourage more girls into science and even calls to address gender biases in the scientific community and for more stable career paths that are amenable to family life.
However, these are short sighted measures if, once in science, women are faced with an all or nothing, ‘publish or perish’ choice. It is not surprising that they choose to leave. The findings of this study are worrying and should be a catalyst for further debate on how to retain and reward talented scientists irrespective of gender. The way their contributions are measured and recognized has to change.
More fundamentally than a ‘publish or perish’ culture is the use of closed pre-publication journal peer review. In this model, editorial readers (including anonymous referees) can hold a weaker relation of accountability to judgements and publishing decisions when these remain invisible and therefore not able to be held to account by a wider audience. I explore these dynamics in my doctoral research. I didn’t include Nobel Laureate Barbara McClintock in my case studies, but she is an example of a women researcher who simply stopped writing at one point (and received a belated Nobel Prize some 40 years later)… though there are arguably many variables to explore in her case.
https://hdl.handle.net/10393/31238 – Gaudet, J. 2014. All that glitters is not gold: The shaping of contemporary journal peer review at scientific and medical journals. uO Research. Pp. 1-23.
https://hdl.handle.net/10393/31238 – Gaudet, J. 2014. How prepublication journal peer review (re)produces ignorance at scientific and medical journals: a case study. uO Research. Pp. 1-67.
https://hdl.handle.net/10393/31161 – : Gaudet, J. 2014. Investigating journal peer review as scientific object of study. uO Research. Pp. 1-11.
I just read most of the first and the third reference below. The outside perspective is interesting, but you miss a few key points. First, reviewer anonymity is important because it allows the reviewer to give honest assessment of the paper without fear of reprisal from the authors. The more problems the paper has, the more important it is that the reviewers feel able to point them out. Open peer review means that reviewers must either be uncritical and avoid upsetting the authors, or point out the flaws and risk making enemies. Since the authors are likely to be the reviewers or editors for that reviewer’s own work, these are bad enemies to make.
Second, you seem to think there is no transparency for editorial judgments, as if the author receives a one word ‘accept/reject’ decision letter. In reality, as best as the editor can manage, the decision is a logical argument based on their own opinion and those of the reviewers. The authors can readily see which aspects of the reviews led the editor to make the decision they did. The identity of the editor is also normally known to the authors, allowing them to dispute the decision if they wish.
Third, the reason journals cost so much is because a quality peer review process is expensive to administer, and for highly selective journals many papers have to be assessed for each one that is accepted. A few sources give the cost of a round of review as about $300, and an editorial reject about $50. A journal with a 10% acceptance rate thus has to spend a lot of money to get to just one accepted paper. All the costs are incurred with paying editorial office staff.
Last, you say in ‘all that glitters…’ (end of section II) that peer review cannot usefully be investigated with the scientific method (in dismissing studies of double blind review that find no effect), but then cite numerous other empirical studies of peer review as evidence that peer review is flawed. You can’t have it both ways.
Tim, thank you for your close readings! Assuming that anonymity is important because it allows reviewers to give honest assessments is a standard argument … unfortunately it is based on an assumptions of honesty and integrity and there is plenty of evidence that even judges who are ‘blinded’ and where there is no transparency… there is potential for corruption. You might enjoy reading Richard Smith’s (former BMJ editor) assessment that even if he knew he would be losing his ‘best’ young referees when removing anonymity (who would probably be impacted in their careers given the circularity of peer review), he was willing to do so to gain transparency and accountability. One of the basic problems here is that a move towards anonymity is performed piece-meal – it would have to be a blanket change in order for it to impact all interrelated aspects of peer review. As it stands, those who gain the most by reifying anonymity do wish to keep it so…
There are several components in editorial communications – referee judgements, editor judgements, and editorial decisions. Yes, the editor is non-anonymous, but the judgement is secret – only between the author and editor. Some journals even make the editorial judgements copyrighted materials that cannot be transmitted or discussed. These are all relevant dynamics.
Your last comment goes back to the first. Fundamentally, when looking at the roots of journal peer review within a legal context – a judge cannot be anonymous based on a need for transparency and accountability. This argument underlies all others. That I cite ‘peer reviewed’ papers is part of academic writing – and that you perceive that I cite only one side of the argument only reflects the fundamental roots in ‘judgement’.
Anonymity only looks ‘normal’ because it has and continues to be constructed as normal. It is difficult for a fish to ‘see’ water 🙂
PS – you might be interested in unabridged versions of my historical shaping piece:
https://hdl.handle.net/10393/31319 – Gaudet, J. 2014. Investigating journal peer review as scientific object of study: unabridged version – Part I. uO Research. Pp. 1-24.
https://hdl.handle.net/10393/31320 – Gaudet, J. 2014. Investigating journal peer review as scientific object of study: unabridged version – Part II. uO Research. Pp. 1-20.
“Assuming that anonymity is important because it allows reviewers to give honest assessments is a standard argument … unfortunately it is based on an assumptions of honesty and integrity and there is plenty of evidence that even judges who are ‘blinded’ and where there is no transparency… there is potential for corruption”
It’s a standard argument because it makes sense, and is borne out by experience. Yes, peer review does assume that the reviewers are attempting to give an honest and objective assessment of the paper, and (as far as I can tell) that’s largely true. That may be different in medicine, where there is much more money and pride involved and open review is more common (which is the field Richard Smith was in)
Because you’re demanding that we get rid of reviewer anonymity for science as a whole and thereby remove one of the protections for reviewer objectivity, you have to demonstrate that reviewers are so dishonest that such a step is necessary. Vague assertions that there is ‘potential for corruption’ isn’t enough.
Moreover, when reviewers are anonymous to the author they are still accountable to the editor. I don’t see that having reviewers reveal their identity to the authors is a significantly greater motivator to be polite and fair than just being identified to the editor. Being identified to the authors does motivate reviewers to soft peddle their opinions, and that has to be actively resisted if they’re to supply an objective review.
Thank you for your reply Tim, interesting indeed. Unfortunately we are holding parallel conversations and are not meeting on common ground. I propose to study journal peer review as a scientific object of study where not one element of its ‘common experience’ cannot be open to investigation and questioning – not based on ‘vague’ assertions but on empirical evidence that does not support ‘honesty’, ‘integrity’, ‘objectivity’ – these are ideals, not attained in peer review (see preprints for empirical and theoretical research supporting these, and thousands of studies on retractions and journal peer review that do not support these). To single out areas of science as ‘more prone to dishonesty’ is an excellent example of looking at multiple and conflicting accountabilities – here to economic actors.
Legal studies help frame ‘ideals’ of honesty, integrity, and objectivity in terms of expected rational decision-making. To obtain maximal potential for rational decision-making requires minimal relational conditions – transparency in judgements and naming the ‘judge’. Investigating journal peer review as a scientific object of study requires indepth understanding of dynamics – and when it is understood as a form of boundary judgement – making judgements as to what is deemed as scientific, or not – there are many practices that simply do not contribute to these decisions but are simply part and parcel of pre-publication journal peer review (see the ignorance (re)production preprint).
Accountability to an editor – is holding an accountability to someone who himself/herself holds multiple accountabilities. These are intersecting and can conflict depending on the goals that each actor in the peer review process wishes to achieve. That these remain ‘invisible’ means that the editor holds absolute judgement… Looking at all accountabilities is imperative – there is no such thing as ‘absolute’ and ‘objective’ scientific knowledge – my research into the purported rejection of new scientific ideas supports that referees make localized judgements based on disciplinary accountabilities and experience that include a relation of accountability to empiricism – and scientific explanation imperatives…
Looking at the editor-referee relationship also detracts from more fundamental questions – how pre-publication journal peer review itself can contribute to potential abuse… Self-regulation in science can hinder instilling practices across the board based on legal premises.
One funding agency in Canada is holding trials on non-anonymous grant referee reporting…
I do not ‘demand’ anything :-), I investigate journal peer review.
Furthermore, to expect that referees require ‘protection’ assumes that they are engaging in accusations that can be ‘true’ or false’… Is this the case? If referees are engaging in rational decision-making with well-founded and well-argued judgements, why would they need ‘protection’?
Have a look at this preprint in which I compare the need to protect witnesses in inquisition to a need to ‘protect’ referees – when making accusations, Gaudet, J. 2014. An end to ‘God-like’ scientific knowledge? How non-anonymous referees and open review alter meanings for scientific knowledge. uO Research. Pp. 1-12. https://hdl.handle.net/10393/31415
You propose “Being identified to the authors does motivate reviewers to soft peddle their opinions, and that has to be actively resisted if they’re to supply an objective review.” Unfortunately these are not borne out with the more than contradictory results from research into masking/unmasking referees… the argument is at the wrong place… what is a referee judgement? What is rational decision-making? What scientific explanation imperatives are referees using in the judgements? Why and how did anonymity enter into boundary judgements from its initial legal roots so as to determine what could be deemed as scientific or not?……
Finally, dynamics of transparency and non-anonymity contribute to greater trust – science cannot but win by gaining this greater trust from actors within, and outside of science.