Glam sticks: the next crystal ball for academic success

A new paper by van Dijk and colleagues claims to have figure out how to predict “success on the academic job market,” according to the title. Given how tight the academic job market is, you might expect this paper to get a lot of downloads.

But the paper’s title promises waffles and the paper delivers pancakes.

If you have a paper that claims to be able to predict success in the job market, you might expect that the authors have measure things like how many people in their pool get tenure-track job offers, how many are awarded tenure, and so on. They don’t measure employment at all.

This paper says it measures the probability of someone becoming a principle investigator. That term is mostly user in reference to grants, so you might think that they are looking through award databases to see what people have gotten grants. They don’t measure granting success, either.

What this paper measures is how people become last author on papers. There is an assumption here that the last author is the “boss” of the paper. This isn’t a bad assumption, but authorship is a compleicated game, and people don’t always follow expected conventions for authorship (Shapiro et al. 1994).

van Dijk and company dug into PubMed, and tracked the publication records of many thousands of individual authors. They were careful to try to remove people who have similar names, so that each record is a single author. Using PubMed as a data source has its limitations, since PubMed’s purpose is to archive biomedical literature. It is an open question whether the patterns van Dijk and colleagues find would hold for other fields, like physics, social sciences, or even non-medical biology.

They find some very interesting patterns. The best predictor of quickly becoming last author on a series of papers is the Impact Factor of the journals you publish in early in your career. Frankly, this is extremely depressing news for those of us interesting in breaking the back of Impact Factor into a thousand pieces, because it shows, “Glam sticks.”

There is hope, though: if you can’t publish in the glamor magazines, you can publish a lot. People could also make the transition to last author by publishing a lot of papers, although the effect wasn’t as big as Impact Factor.

The authors also show that – to almost nobody’s surprise – it helps to be a guy. And it helps to have a degree from a fancy university. I’m still trying to work out if there are any confounds in their discussion of university rankings, however. The universities are partly ranked using the same publication metrics that they use elsewhere in the paper, so it’s not surprising that the university rankings are correlated with other features they include in their prediction algorithm.

If you are willing to accept the authors’ peculiar definition of “PI” as a proxy for “academic success,” there is a lot of interesting stuff in this paper.

Additional, 3 June 2014: If you want to play around with the authors’ prediction algorithm, they have a website called PI Predictor. It is a bit reassuring to know that the odds were in my favour.

Also, I’m quoted in this Nature News piece about the article.

Reference

Shapiro DW, Wenger NS, Shapiro MF 1994. The contributions of authors to multiauthored biomedical research papers. JAMA 271: 438-442. http://dx.doi.org/10.1001/jama.1994.03510300044036

van Dijk D, Manor O, Carey LB 2014. Publication metrics and success on the academic job market. Current Biology 24(11): R516-R517. DOI: 10.1016/j.cub.2014.04.039

External links

Van Noorden R. 2014. Computer model predicts academic success. Nature. http://dx.doi.org/10.1038/nature.2014.15337
Bohannon J. 2014. Science Moneyball: The Secret to a Successful Academic Career
Bohannon J. 2014. Want to Be a PI? What Are the Odds? Science Careers http://dx.doi.org/10.1126/science.caredit.a1400136
Bohannon J. 2014. Career moneyball. Science Careers. http://dx.doi.org/10.1126/science.caredit.a1400135