A light touch or hands on? Monday morning quarterbacking a retraction

Things moved fast last week. I blogged about a paper with strange references to “The Creator”. Within hours, the story exploded on social media (the main hashtag was #HandOfGod on Twitter), and even garnered some tech news coverage before the paper was retracted. I was teaching and traveling when a lot of this went down, so didn’t have much change to follow up until now.

PLOS ONE deserves credit for not dawdling on this. A concern was identified, and by the next day, they arrived at a decision.

PLOS ONE loses the points they just earned, however, for the lack of transparency in their retraction notice.

Our internal review and the advice we have received have confirmed the concerns about the article and revealed that the peer review process did not adequately evaluate several aspects of the work.

Which means... what, exactly? PLOS ONE claims it will publish any technically sound science. What’s the basis for retraction, then? What about the paper is not technically sound? You can’t find it in the retraction notice. I haven’t seen any criticisms about the paper’s scientific content beyond, “It’s not news; we know hands grab things.” But no comments like, “The wrong statistic was used,” or “The authors didn’t account for this uncontrolled variable.”

The retraction notice says the editorial process wasn’t very good, but that should not be the basis for a retraction. If an editor screwed up, a competent paper shouldn’t suffer. A good result shouldn’t be punished for a bad process.

PLOS staffer David Knutsen wrote more for this blog post, but we still only get vague references to:

(T)he quality of the paper in general, the rationale of the study and its presentation relative to existing literature(.)

It still doesn’t answer what technical flaw the paper had. PLOS ONE comes out looking like it might have swatted a flay with a sledgehammer, and facing accusations of racism.

This situation is not a bug in the PLOS ONE system: it’s a feature. Ph.D. Comics published this cartoon about PLOS ONE back in 2009, which takes on new relevance:

“Let the world decide what is important.” PLOS ONE was designed from the get go to:

1. Have a light editorial touch. “Is it technically competent? Then publish.” The high volume of papers (PLOS ONE is the biggest single scientific journal in the world) pretty much ensures you cannot have the same level of editing that happens at other journals.
2. Encourage post-publication peer review.

When you combine those two things, things like the “hand of god” paper are practically inevitable. The question becomes whether the benefits of light editing (authors having more direct and potentially faster communication of what they want) outweighs the potential downsides (disproportionate criticism for obvious things that could have been fixed).

Related posts

A handy new Intelligent Design paper

External links

Follow-up notification from PLoS staff
A Science Journal Invokes ‘the Creator,’ and Science Pushes Back
Hand of God paper retracted: PLOS ONE “could not stand by the pre-publication assessment”
The “Creator” paper, Post-pub Peer Review, and Racism Among Scientists
This Paper Should Not Have Been Retracted: #HandofGod highlights the worst aspects of science twitter