This is a guest post by Eve Gray. Once an academic publisher in Cape Town, she went on to be a publishing strategy consultant before becoming a researcher on open access scholarly communications at the Centre for Innovation in Teaching and Learning and the IP Law Unit at UCT. To mark the release of the 2014 Impact Factor Report she debates the value impact factors serve for the developing world.  

The impact factor under fire

The release of the 2014 Impact Factor Report was being awaited, as usual, with some anticipation by journal publishers and researchers to see who is in and who is out in this particular club this year. Yet this comes at a time when there is an ever-rising tide of contestation about its value as an analytic tool for research effectiveness in a radically changing research environment, and especially in the developing world. Among many others, Stephen Curry went viral in 2012 with his stinging dismissal of the IF as ‘statistically illiterate’.

The Impact Factor and the developing world

There is an interesting circularity about the impact story in the developing world. With the expansion of the number of developing country journals in the index, the inclusion of the Latin American open access journal platform, SciELO in the Wed of Science (the Thomson Reuters citation indexes), it would seem that there is a courtship going on in which the developing world is being drawn into the journal impact tables.

This is a two-way process, as has recently been analyzed in the Latin American context by Vessuri, Guédon and Cetto where they express concern that the search for ‘international’ status for SicELO journals through a chase for impact factors tends to work against development agendas, giving rise to serious concerns about equity.

A system of exclusion

The IF is based on scholar-to-scholar communication in journal articles alone. Its version of relevance and excellence is designed to exclude research on most of the critical development issues that face the global South. The Editor of The Lancet, Richard Horton, quoted in Times Higher Education in 2009, claiming that if he published more work from Africa rather than research on clinical trials in rich countries, the Lancet’s citation and impact factor would probably go down:

“The incentive for me is to cut off completely parts of the world that have the biggest health challenges … citations create a racist culture in journals’ decision-making and embody a system that is only about us (in the developed world).”

Open access

Open access journals have been much readier to redefine what constitutes impact and to address the inequalities and exclusions of the current system. There is a greater willingness in journals like BioMed Central, to engage with the developing world and to provide some publishing space for neglected diseases, for example. The development of megajournals, like PLOS ONE, has reconceptualized the concept of impact, divorcing it from the idea of research quality and providing space for a longer tail of research products. The question is whether this model could be adopted in the developing world, and, if so, what sustainability model it would need to rely on.

With Brazil now the second largest open access journal publisher in the world, there is also hope for the increased visibility and impact for other countries, using open access. The question is, rather, what kind of impact is needed.

Prestige and relevance

The findings of the Scholarly Communication in Africa Programmes (SCAP) mapped research and communication practices in four southern African countries, providing insight into the nature of impact in this context. It was summed up by a senior research administrator, who said, ‘The greatest challenge we face in delivering research impact is to combine prestige and relevance.’

Given the invisibility of African publications in the journal indexes, there has been a general assumption that little research is being produced in the region. This is compounded by a failure to take a strategic approach to research communication, nor take advantage of the Web 2.0 technologies to broaden the reach of the work produced in the region. But, most of all, it is the failure of a reliance on the citation indexes to record what is really being produced that fuels this apparent vacuum.

Cameron Neylon of PLOS summed it up in a blog posting after his first visit to the SCAP project in Cape Town:

‘I spend a lot of time in the UK and the US trying to convince people that there is an issue; that we need to look at how our research matters to the wider community. In South Africa the needs are clear, and scholars want to make a difference. The question is how, and how to tell when it is working.  They are damn good at this. We could learn a lot from these people about how to balance the pursuit of prestige and “research with impact” that seems to be such a struggle for us.’

He might be right: there appears to be an increasing focus in the North on the need for evidence-based research and the different kind of outputs and the different communication strategies that are needed for this.

‘This is witnessed, for example, in the European Commission’s push for greater inclusivity through open science, epitomized in the call by Vice-President Neelie Kroes in her speech at the Science 2.0 Conference in March 2014, where she argued forcefully for a change in the evaluation of scholarship, a move away from the current metrics of ‘citation counts and the number of publications’, to take into account ‘teaching, outreach, communication and collaboration’.