Back, um, “some time ago” (six years now crap I’m old), I dealt with the question of whether peer reviewers are overburdened. That is, there are too many papers and not enough people willing to review them all. At the time, I was suspect of the claims that being asked to review one to three papers a month was normal.
This question came up again on Twitter today. Since it’s been a few years, I wondered if I was still a pariah. I got the impression I was being asked to review more now...
And I am. The trendline is definitely upward. But it’s still far less than the “one to three papers a month” figure that people were claiming. I might hit the “one per month” around 2022.
I was part of the reviewer “talent pool” in the early 2000s, but got very few invites. I am the same guy now as then, so what’s changed? I think I’m getting asked to do more reviews because of the time spent in the academic system. And I was, luckily, able to step up my own publication game around 2010, which may have contributed to my “name visibility” among editorial boards.
I know some people on Twitter who are on journal editorial boards, and they do indeed complain about finding reviewers. But I wonder how well editors use the available talent pool. I would bet that journal peer review invitations are biased against:
- Faculty who are not at American universities. (Update, 21 November 2016: Warne (2016) reports proportionately more peer review is performed by American researchers than Chinese ones.)
- Faculty who are not at English-speaking universities.
- Faculty at undergraduate institutions.
- Post-doctoral fellows and graduate students.
- Women reviewers.
- Minority reviewers.
Update, 21 November 2016: Table 1 in Okike et al. (2016) shows more than ten men for every one woman reviewing manuscripts. Hat tip to Laura Jurgens.
Update, 5 December 2016: This tweeted list of “Top reviewers” from the journal Neurospsychopharmacology has nine men and one woman. Hat tip to Bita Maghaddam.
Update, 23 March 2017: Big new study in eLife by Helmer and coleagues (2017) supports that the hypothesis that “women are underrepresented in the peer-review process.”
Helmer M, Schottdorf M, Neef A, Battaglia D. 2017. Gender bias in scholarly peer review. eLife 6: e21718. http://dx.doi.org/10.7554/eLife.21718
Okike K, Hug KT, Kocher MS, Leopold SS. 2016. Single-blind vs double-blind peer review in the setting of author prestige. JAMA 316(12): 1315-1316. http:/dx.doi.org/10.1001/jama.2016.11014
Warne V. 2016. Rewarding reviewers – sense or sensibility? A Wiley study explained. Learned Publishing 29(1): 41-50. http://dx.doi.org/10.1002/leap.1002
Peer review pariah