Initial reactions: can we hide the sex of authors? And should we?

Maliniak and colleagues (2013) have found an important fact: if a paper in international relations has only female authors, it will be cited about 70% as often as a paper with only male authors.

In a guest blog post, co-author on that study, Barbara Walter, says there is a “relatively easy” solution to fix this. I was reminded of the saying, “For every complex problem there is a simple solution, and it’s wrong.”

(There’s a) potentially easy solution: anonymity. What if we set up evaluations in academia so that we never knew the gender of the person being evaluated (or at the very least downplayed it as much as possible)?

Under this suggestion, I should be “Z. Faulkes,” always, in my academic writing.

I am sympathetic to the idea of removing identifiers. After all, if something is supposed to be judged on its content, then let’s focus on the content. But what bugs me is that the authors say this is “relatively easy.” Indeed, the example they use makes it sound easy:

Prior to adopting anonymous auditions in which candidates sat behind screens when they played, less than 5 percent of all musicians in these orchestras were women. Once blind auditions were instituted women were 50 percent more likely to make it out of the preliminary rounds and significantly more likely to ultimately win the job. Today, 25 percent of all musicians in these elite orchestras are women.

The more I thought about it, the more the analogy with a musical audition unraveled.

A musical audition asks different performers to perform a set, predetermined piece of music. In academic evaluation, researchers are submitting original works, and no two are alike.

A musical audition is a solo endeavor, where you can judge the performance of the individual. In academics, particularly in science, you are often working with many authors, and it’s very difficult to disentangle the contributions of the authors.

Evaluating academics isn’t like auditioning a single soloist in an orchestra. It’s more like digging through demo tapes of rock bands. And you may have heard the band before at a bar or showcase gig.

For academics, their manuscripts and their CVs are extensively reviewed.

Double blind reviews of single manuscripts has been tested occasionally. For instance. Budden and colleagues (2008) claimed this increased the number of papers with women as the first author. Those results were contested, however (Webb et al 2008; Whittaker 2008). But even the authors of the original study admitted it was incredibly difficult to get data on the effects of different peer-review practices, since so few journals are willing to tinker with it, or share data about it.

As a practical matter, I don’t think there is any way to make a CV even close to anonymous. Manuscripts are perhaps a more promising target for being reviewed anonymously. The nature of peer review, trying to find the most qualified people to review papers with the most expertise, makes it highly likely that reviews will be done by people who have met the authors and know what their research is.

For instance, early in my career, I got wind that my name had come up in an Animal Behavior Society meeting. Someone (I think it was Zuleyma Tang-Martinez) was making the point that the society should know the sex of all of its members, “For instance, is this ‘Zen Faulkes’ male or female?”

As it happened, not one, but two of my professors from my university were in the room. They laughed, and said, “Male... You should have asked!”

For that matter, making a work anonymous will most likely change the “females cited 70% as often as males” if the problem is bias at the reviewing stage. It is unlikely to address the citation problem that is coming post review, like males being more active in self-promotion and networking.

A few years ago at Science Online, Ed Yong mentioned that he often had scientists emailing him, asking if Ed could mention their newest work. These were invariably men. The women in the room reacted with surprise. None of them had thought that this was something that they could or should do. Whether you think such requests are out of line or not, you could see how that kind of difference in promotion could lead to men being better known in their field, and getting some more citations as a result.

That the scientific community is a community means that anonymity will be difficult to come by except in very early careers. Double blind review might be much more practical in situations like applications to grad school.

Let’s fight biases whenever we come across them. but let’s not pretend it’s going to be easy. It’s going to be a tough, hard slog.

Hat tip to Miriam Goldstein for bringing this to my attention.

Update, 16 May 2015: Any discussion of making peer review blinded or not is probably incomplete without referencing this post by Hilda Bastian, which systematically reviews a big chunk of the literature on effects of anonymity in peer review. She concludes:

I think institutionalizing anonymity in publication peer review is probably going out on a limb. It’s only partially successful at hiding authors’ identities, and mostly only when people in their field don’t know what authors have been working on. If blinding authors was a powerful mechanism, that would be evident by now.

Update, 21 September 2022: Another paper claiming to show bias in peer review made the rounds recently on Twitter, which again prompted many people to argue, “Blinding manuscripts is the answer, obviously.” Hilda Bastian once more enters the fray with another deep read and analysis of the paper. A few choice points from her post.

An effect on peer reviewers’ opinions isn’t the same thing as an effect on publication bias.

Good editors matter.

Could this manuscript be an inherently divisive one? We can’t know, because no evaluation of the manuscript by methodological and content experts was reported, and the expertise of the peer reviewers for the particular subject matter wasn’t reported either.

We are being asked to take a lot on trust here.

Hilda concludes that this paper is still relatively unconvincing in making a case for blinded peer review. She notes that a big study is in the works and should be out soon. But I recommend reading her whole post.

References

Maliniaka D, Powers R, Walter BF. 2013. The gender citation gap in international relations. International Organization 67(04): 889-922. DOI: http://dx.doi.org/10.1017/S0020818313000209

Budden AE, Tregenza T, Aarssen LW, Koricheva J, Leimu R, Lortie CJ. 2008. Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution 23(1): 4-6. http://dx.doi.org/10.1016/j.tree.2007.07.008

Webb TJ, O’Hara B, Freckleton RP. 2008. Does double-blind review benefit female authors? Trends in Ecology & Evolution 23(7): 351-353. http://dx.doi.org/10.1016/j.tree.2008.03.003

Whittaker RJ. 2008. Journal review and gender equality: a critical comment on Budden et al. Trends in Ecology & Evolution 23(9) 478-479. http://dx.doi.org/10.1016/j.tree.2008.06.003

Budden AE, Aarssen LW, Koricheva J, Leimu R, Lortie CJ, Tregenza T. 2008. Response to Whittaker: challenges in testing for gender bias. Trends in Ecology & Evolution 23(9): 480-481. http://dx.doi.org/10.1016/j.tree.2008.06.004

External links

How to reduce the gender gap in one (relatively) easy step
The problem is that scientists are human
Weighing up anonymity and openness in publication peer review
Photo by Penn Provenance Project on Flickr; used under a Creative Commons license.