Strange Things

I noticed something strange about the 2013 Impact Factor data for eLife.

Before I get onto the problem. I feel I need to point out that I dislike Impact Factors and think that their influence on science is corrosive. I am a DORA signatory and I try to uphold those principles. I admit that, in the past, I used to check the new Impact Factors when they were released, but no longer. This year, when the 2013 Impact Factors came out I didn’t bother to log on to take a look. A chance Twitter conversation with Manuel Théry (@ManuelTHERY) and Christophe Leterrier (@christlet) was my first encounter with the new numbers.

Huh? eLife has an Impact Factor?

For those that don’t know, the 2013 Impact Factor is worked out by counting the total number of 2013 cites to articles in a given journal that were published in 2011 and 2012. This number is divided by the number of “citable items” in that journal in 2011 and 2012.

Now, eLife launched in October 2012. So it seems unfair that it gets an Impact Factor since it only published papers for 12.5% of the window under scrutiny. Is this normal?

I looked up the 2013 Impact Factor for Biology Open, a Company of Biologists journal that launched in January 2012* and… it doesn’t have one! So why does eLife get an Impact Factor but Biology Open doesn’t?**

elife-JIFLooking at the numbers for eLife revealed that there were 230 citations in 2013 to eLife papers in 2011 and 2012. One of which was a mis-citation to an article in 2011. This article does not exist (the next column shows that there were no articles in 2011). My guess is that Thomson Reuters view this as the journal existing for 2011 and 2012, and therefore deserving of an Impact Factor. Presumably there are no mis-cites in the Biology Open record and it will only get an Impact Factor next year. Doesn’t this call into question the veracity of the database? I have found other errors in records previously (see here). I also find it difficult to believe that no-one checked this particular record given the profile of eLife.

elfie-citesPerhaps unsurprisingly, I couldn’t track down the rogue citation. I did look at the cites to eLife articles from all years in Web of Science, the Thomson Reuters database (which again showed that eLife only started publishing in Oct 2012). As described before there are spurious citations in the database. Josh Kaplan’s eLife paper on UNC13/Tomosyn managed to rack up 5 citations in 2004, some 9 years before it was published (in 2013)! This was along with nine other papers that somehow managed to be cited in 2004 before they were published. It’s concerning enough that these data are used for hiring, firing and funding decisions, but if the data are incomplete or incorrect this is even worse.

Summary: I’m sure the Impact Factor of eLife will rise as soon as it has a full window for measurement. This would actually be 2016 when the 2015 Impact Factors are released. The journal has made it clear in past editorials (and here) that it is not interested in an Impact Factor and won’t promote one if it is awarded. So, this issue makes no difference to the journal. I guess the moral of the story is: don’t take the Impact Factor at face value. But then we all knew that already. Didn’t we?

* For clarity, I should declare that we have published papers in eLife and Biology Open this year.

** The only other reason I can think of is that eLife was listed on PubMed right away, while Biology Open had to wait. This caused some controversy at the time. I can’t see why a PubMed listing should affect Impact Factor. Anyhow, I noticed that Biology Open got listed in PubMed by October 2012, so in the end it is comparable to eLife.

Edit: There is an update to this post here.

Edit 2: This post is the most popular on Quantixed. A screenshot of visitors’ search engine queries (Nov 2014)…

searches

The post title is taken from “Strange Things” from Big Black’s Atomizer LP released in 1986.

Advertisements

15 responses

  1. Very interesting post. Like you I dislike IF and naturally signed DORA.

    eLife are not listed in the new Impact Factors http://library.kmu.ac.ir/Images/UserUpload/Document/Library/jcr_impact_factors_list_2013.pdf

    I’ve never heard of partial Impact Factor until today and there is hardly any information on the web about it.

    Aside from that, clarification from eLife has been requested…

    1. Thanks for the comment. The data for eLife pops up on JCR when you search by journal name… which is – I suspect – what University Administrators will do.

  2. And from yesterday, further proof (as if it was needed) that the output from Thompson Reuters should not be trusted. http://metro.co.uk/2014/08/17/company-doesnt-quite-understand-venn-diagrams-after-producing-one-that-disparages-it-4835783/

    1. That is hilarious. Funnier if it is an Euler plot rather than Venn diagram #MathsJoke

  3. There is one miscite from a 2013 eLife paper to an inexistent 2010 eLife paper, and another miscite from a 2013 PLoS Computational Biology paper to an inexistent 2011 eLife paper.

    1. Thanks. Mystery of the miscitations solved.

  4. […] off the heels of that meeting just two weeks ago, I was surprised to see that eLife was granted an Impact Factor this year. Having only started publishing issues in late 2012, how could they have a 2013 Impact Factor? […]

  5. […] post on the strange data underlying the new impact factor for eLife was read by many people. Thanks for the interest and for the comments and discussion that followed. […]

  6. I noticed a systematic and large scale difference between the report from the Thomson Reuters Corporation and SJR: http://www.scimagojr.com/journalsearch.php?q=21100242814&tip=sid&clean=0
    For most journals, including young journals such as Cell Reports, the 2-year citations per article are quite close to the reported impact factor. However, for eLife the difference is huge, even when correct for the fraction of uncitable articles published by eLife.

    1. That is strange. I think the main point here is that these calculations should only be done for journals that have been publishing throughout the survey period.

  7. […] most popular post on the blog (by a long way) is “Strange Things“, a post about the eLife impact factor (2824 views). The next most popular is a post about a […]

  8. […] be more open with the data (although I’ve not seen any evidence for this). I wrote about partial impact factors instead, which took over my blog. Anyway, the analysis shown here is likely to be similar for any year […]

  9. […] previously (here and here). The most popular posts are those on publishing: preprints, impact factors and publication lag times, rather than my science, but that’s OK. There is more to come on […]

  10. […] off the heels of that meeting just two weeks ago, I was surprised to see that eLife was granted an Impact Factor this year. Having only started publishing issues in late 2012, how could they have a 2013 Impact Factor? […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: