Impact and the rearview mirror

- 0 Comments

Impact factors are like intelligence quotients: it’s clear they are a measure of something, but it’s not entirely clear what. BMC Biology’s has gone up again this year, as has its ranking, and whatever one may think about impact factors, they are, at least for the time being, important to the community we serve, and for that reason if for no other (and there are of course others), this is to be welcomed for the benefit to the authors whose articles we have published.

 Volumes have already been written on the quirks of impact factor calculations and the absurdity of the two-year impact factor of the journals he/she publishes in as a measure of an individual’s worthiness for a position, a grant or a promotion. Among many objections to this criterion of worthiness is its short-sightedness: two years is too short a time to have any serious notion (for the vast majority of papers) of the importance of a research publication. A particularly articulate contribution to this argument has been published recently by Marc Kirschner, writing in Science on the recent  introduction by the US National Institutes of Health of impact and significance as criteria for the funding of research. Impossible to judge, except in cases where no significantly new ground is being broken. Kirschner cites the outstanding case of restriction enzymes, whose gargantuan impact on the whole of modern biology could not possibly have been guessed when they were first established as a bacterial defense against phage.

 There are some nice examples from more recent research. Ralph DeBerardinis and colleagues developed a technique critical for characterizing the metabolic programming of human tumour cells that he was unable to publish in any of the better-recognized journals, though in this case one would have thought it might have been clear that a method of tracking the Warburg effect in tumour cells could have significant impact. Christopher Dobson’s seminal work on protein misfolding disorders grew out of studies on lysozyme mutants, which seemed of relatively limited interest until it turned out that they could form systemic amyloid deposits, and led to the realization that many (or most) proteins can form amyloid if their folding kinetics are disrupted (by mutation or cellular conditions), offering a new approach to intervention in protein misfolding diseases.

 It seems more to the point to judge research on its quality than to try to make guesses about its significance and impact – and, of course, to be willing to take a chance on interesting observations that may or may not lead somewhere.