When assessing the impact of a published research article, it might seem logical to look at the Impact Factor of the journal that you find it in. But as journals and scholars have moved online, and citation indexing has been automated, the wealth of information for citation discovery and analysis has vastly increased.

Citation counts can tell a more accurate story about the scholarly impact that an individual article has made than the journal Impact Factor. But where should you look for these counts? There are a number of indexing services tracking and providing information about citations, each with advantages and disadvantages.

To highlight just how different the citation counts can be between the different indexes, we’ve had a look at some of the highly cited content across Springer and BioMed Central’s extensive journal portfolios in this area to tell us a bit more.

Comparing the average citation counts to a sample of 33 highly cited articles across ISI Web of Science, Scopus, PubMed Central (PMC), CrossRef and Google Scholar reveals that the numbers, certainly in the area of stem cell research, differ quite substantially.

As the graphic below shows, the average number of citations to the articles we looked at is highest in Google Scholar, with over three times the number of citations indexed in PubMed Central.

One sample article that we looked at in more detail had over 5 times more citations in Google Scholar than PMC.

So what’s the reason for the difference?

There are in excess of 400 abstracting and indexing services which index journals, and the varying citation counts that they display for individual articles can largely be attributed to differences in the sources that they scour for citations.

The services might look at different areas of research, for example Chemical Abstract Services index approximately 8,000 journals of chemical information, whereas MEDLINE covers around 5,000 journals in life sciences and biomedicine. It’s clear that the number of citations to an article relevant to both fields will differ depending on which you look at, as the set of journals in each service won’t hugely overlap.

Even when indexing services are covering journals in similar disciplines, they might not necessarily include the same ones. Web of Science covers over 11,000 journals in areas including science, social science, arts and humanities, but Scopus (which covers similar subject areas) indexes around 18,000. You’d probably expect to see higher average citation counts reported by Scopus, which is certainly reflected in our small sample of articles.

This can be accounted in part to the criteria that a source has to meet before it will be indexed. The services that tend to be more selective will necessarily index fewer journals, perhaps accounting for the discrepancy between the number of sources indexed by Web of Science and Scopus (more information on their selection processes can be found here and here).

Another difference is the type of source covered. Google Scholar covers theses and selected web pages in addition to the journal, books and conference proceedings coverage common to the rest. CrossRef also includes data sets.

Which source should I use?

There are benefits to each of course – if you’re looking to find the most complete overview of the scholarly products that cite an article then Google Scholar might provide the highest number of citations, but might not filter the citing content for high quality, authoritative sources as Web of Science claims to do.

When comparing the ‘impact’ of different articles it’s probably best to compare like with like, and use the same index.

To make the discovery of articles citing our publications easy, BioMed Central articles have links to various indexing services from the ‘Article metrics page’ of each publication, and Springer.com provides the number of citations via CrossRef, and a links to a page which brings them together.

For more information, and free access to the Springer articles until July 28th, visit this page.