NeuroDojo: Editors retract paper claiming GM food caused cancer in rats

29 November 2013

Editors retract paper claiming GM food caused cancer in rats

The wheels grind slowly, but they do grind.

Last year, a paper by Séralini and colleagues came out that claimed eating genetically modified (GM) food caused cancer in rats. The online reaction was swift and sharp: the samples were too small, too many data were not shown, the statistics were odd or missing. The reaction that made it to the journal’s printed pages was almost equally critical.

The journal editors have now unilaterally retracted the paper, Nature reports.

The journal editor went back, got the raw data, and had the paper reviewed again. The press release claims the small sample size was seen as a shortcoming of the paper in the first round of reviews from peer-review... unfortunately, given that the journal follows the normal editorial process of keeping all reviews secret, there is no way to verify this claim. We have no idea how many reviewers saw the paper, who they were, or what they said.

The press release says:

The peer review process is not perfect, but it does work.

What I would have liked to read following that statement was something to indicate that peer-review could be improved. This is a good example of how peer-review could be improved easily by making peer review more transparent. Publishing the reviews with the paper, for instance.

The press release goes on:

Ultimately, the results presented (while not incorrect) are inconclusive, and therefore do not reach the threshold of publication for Food and Chemical Toxicology.

Interesting. I am not sure whether “conclusive” results should be the threshold for publication. I have never seen “conclusive” listed in the requirements for publication in any journal. Indeed, Food and Chemical Toxicology’s own guidelines don’t include such words:

Papers submitted will be judged on the basis of scientific originality and contribution to the field, quality and subject matter.

If “conclusive” is truly a criterion for publication, we should probably have much more stringent criteria. We should probably demand larger sample sizes or insist on power analyses, lower p values from 0.05 to 0.005, and demand independent replication from another lab before publishing.

The press release doesn’t talk about of the important role that the online community played in drawing attention to this paper. No, it was only the published letters in the journal that were valuable:

Likewise, the Letters to the Editor, both pro and con, serve as a post-publication peer-review. The back and forth between the readers and the author has a useful and valuable place in our scientific dialog.

I get the distinct notion that Elsevier would rather not acknowledge the online community. I wonder if the editors would have gone back and re-reviewed the paper were it not for the Internet shitstorm that erupted?

I’m interested that the press release, in the quote above, specifically points out that there were “pro” letters. But this is classic case of “false balance,” a rhetorical trick to make a one-sided situation look less like a rout; the vast majority of the letters they themselves published were highly critical.

When the paper came out, I argued against retracting it:

The anti-GM would revel in the retraction, and see it as proof of a cover-up by the establishment.

The initial reaction from the authors suggest my prediction was correct:

Séralini and his team... allege that the retraction derives from the journal’s editorial appointment of biologist Richard Goodman, who previously worked for biotechnology giant Monsanto for seven years.

I will be following the reaction to see if my take on this was correct. For a certain kind of conspiracy-minded individuals, I suspect this will be the sort of move that constitutes “proof” of nefarious efforts to hide evidence, rather than clumsy moves to “correct” the “scientific record.”

Additional: In Forbes, a surprisingly comprehensive article for appearing so soon after the news broke. Also shows anti-GM food advocates are not saying, “This is normal science”:

Claire Robinson, editor of the anti-GMO activist site GM Watch and a separate website set up to promote the Séralini study, GMO Seralini, blasted the move as “illicit, unscientific, and unethical,” the first salvo in what will no doubt be a vigorous defense of the study in the weeks ahead.

However, in fairness to Robinson, she raises an excellent point, which echoes mine above:

Robinson also claimed that the retraction violated scientific guidelines laid out by the Committee on Publication Ethics (COPE). COPE guidelines state that the grounds for a journal to retract a paper are: (1) clear evidence that the findings are unreliable due to misconduct (e.g. data fabrication) or honest error; (2) Plagiarism or redundant publication; or (3) Unethical research.

Note that “inconclusive” is not a reason to retract a paper.

More additional: Retraction Watch covers this, and the comment thread is lively.

Additional, 1 December 2013: P.Z. Myers characterizes the retraction as “belated”. This raises a question: what is the right speed for a retraction? Retracting the paper as soon as criticisms appeared (which was was the journal posted the manuscript as “in press”) would have been rash. While I might fault the editorial board for many things, they did due diligence on investigating the paper before deciding whether to retract it or not.

When you visit the article on Science Direct today, however, there is no indication of the retraction. Why put out a press release about the retraction without immediately putting the retraction into effect and marking the paper accordingly?

Update, 6 December 2013: The Ecologist blog reports on a group letter to the Food and Chemical Toxicology editor, which contains this high-handed passage:

This arbitrary, groundless retraction of a published, thoroughly peer-reviewed paper is without precedent in the history of scientific publishing, and raises grave concerns over the integrity and impartiality of science.

“Arbitrary”? When many people call for a paper to be retracted, I don’t think you can call it entirely arbitrary.

“Groundless”? The letter itself describes the paper as “published, thoroughly peer-reviewed paper”, but it doesn’t go out of its way to say the paper is any good. As far as I can tell, and in the opinions of many, the paper is deeply flawed. Also, given that the authors did not specify conflicts of interest, there is a case for it to be retracted for being unethical, although the editor did not take that position.

“Without precedent”? That’s overkill. One of the first times I wrote about retraction was for a paper for which there were no concerns about the quality of the science. There was also this controversial statement statement from PLOS ONE: