The wheels grind slowly, but they do grind.
Last year, a paper by Séralini and colleagues came out that claimed eating genetically modified (GM) food caused cancer in rats. The online reaction was swift and sharp: the samples were too small, too many data were not shown, the statistics were odd or missing. The reaction that made it to the journal’s printed pages was almost
equally critical.
The journal editors have now unilaterally
retracted the paper,
Nature reports.
The journal editor went back, got the raw data, and had the paper reviewed again. The
press release claims the small sample size was seen as a shortcoming of the paper in the first round of reviews from peer-review... unfortunately, given that the journal follows the normal editorial process of keeping all reviews
secret, there is no way to verify this claim. We have no idea how many reviewers saw the paper, who they were, or what they said.
The press release says:
The peer review process is not perfect, but it does work.
What I would have liked to read following that statement was something to indicate that peer-review
could be improved. This is a good example of how peer-review could be improved
easily by making peer review more transparent. Publishing the reviews with the paper, for instance.
The press release goes on:
Ultimately, the results presented (while not incorrect) are inconclusive, and therefore do not reach the threshold of publication for Food and Chemical Toxicology.
Interesting. I am not sure whether “conclusive” results should be the threshold for publication. I have never seen “conclusive” listed in the requirements for publication in any journal. Indeed,
Food and Chemical Toxicology’s
own guidelines don’t include such words:
Papers submitted will be judged on the basis of scientific originality and contribution to the field, quality and subject matter.
If “conclusive” is truly a criterion for publication, we should probably have much more stringent criteria. We should probably demand larger sample sizes or insist on power analyses, lower p values from 0.05 to 0.005, and demand independent replication from another lab before publishing.
The press release doesn’t talk about of the important role that the online community played in drawing attention to this paper. No, it was only the published letters in the journal that were valuable:
Likewise, the Letters to the Editor, both pro and con, serve as a post-publication peer-review. The back and forth between the readers and the author has a useful and valuable place in our scientific dialog.
I get the distinct notion that Elsevier would rather not acknowledge the online community. I wonder if the editors would have gone back and re-reviewed the paper were it not for the Internet shitstorm that erupted?
I’m interested that the press release, in the quote above, specifically points out that there were “pro” letters. But this is classic case of “false balance,” a rhetorical trick to make a one-sided situation look less like a rout; the vast majority of the letters
they themselves published were highly critical.
When the paper came out, I
argued against retracting it:
The anti-GM would revel in the retraction, and see it as proof of a cover-up by the establishment.
The initial reaction from the authors suggest my prediction was correct:
Séralini and his team... allege that the retraction derives from the journal’s editorial appointment of biologist Richard Goodman, who previously worked for biotechnology giant Monsanto for seven years.
I will be following the reaction to see if my take on this was correct. For a certain kind of conspiracy-minded individuals, I suspect this will be the sort of move that constitutes “proof” of nefarious efforts to hide evidence, rather than clumsy moves to “correct” the “scientific record.”
Additional: In
Forbes, a surprisingly comprehensive article for appearing so soon after the news broke. Also shows anti-GM food advocates are not saying, “This is normal science”:
Claire Robinson, editor of the anti-GMO activist site GM Watch and a separate website set up to promote the Séralini study, GMO Seralini, blasted the move as “illicit, unscientific, and unethical,” the first salvo in what will no doubt be a vigorous defense of the study in the weeks ahead.
However, in fairness to Robinson, she raises an excellent point, which echoes mine above:
Robinson also claimed that the retraction violated scientific guidelines laid out by the Committee on Publication Ethics (COPE). COPE guidelines state that the grounds for a journal to retract a paper are: (1) clear evidence that the findings are unreliable due to misconduct (e.g. data fabrication) or honest error; (2) Plagiarism or redundant publication; or (3) Unethical research.
Note that “inconclusive” is not a reason to retract a paper.
More additional: Retraction Watch covers this, and
the comment thread is lively.
Additional, 1 December 2013: P.Z. Myers characterizes the retraction as “
belated”. This raises a question: what is the right speed for a retraction? Retracting the paper as soon as criticisms appeared (which was was the journal posted the manuscript as “in press”) would have been rash. While I might fault the editorial board for many things, they did due diligence on investigating the paper before deciding whether to retract it or not.
When you visit the article on Science Direct today, however, there is
no indication of the retraction. Why put out a press release about the retraction without immediately putting the retraction into effect and marking the paper accordingly?
Update, 6 December 2013: The Ecologist blog reports on a
group letter to the Food and Chemical Toxicology editor, which contains this high-handed passage:
This arbitrary, groundless retraction of a published, thoroughly peer-reviewed paper is without precedent in the history of scientific publishing, and raises grave concerns over the integrity and impartiality of science.
“Arbitrary”? When
many people call for a paper to be retracted, I don’t think you can call it
entirely arbitrary.
“Groundless”? The letter itself describes the paper as “published, thoroughly peer-reviewed paper”, but it doesn’t go out of its way to say the paper is any good. As far as I can tell, and in the opinions of many, the paper is deeply flawed. Also, given that the
authors did not specify conflicts of interest, there is a case for it to be retracted for being unethical, although the editor did not take that position.
“Without precedent”? That’s overkill. One of the first times I wrote about retraction was for a paper for which there were no concerns about the
quality of the science. There was also this controversial statement
statement from
PLOS ONE:
If a paper’s major conclusions are shown to be wrong we will retract the paper.
That’s close to what the
Food and Chemical Toxicology editor is doing, it seems to me. There are probably many other examples.
Update, 11 December 2013: The editor has a
response to the decision to retract the paper. It is an interesting peek “behind the curtains” of the editorial process. In particular, the editor says the guidelines for retraction were followed, answering Claire Robinson’s concern in
Forbes. Hat tip to
Nathanael Johnson.
Related posts
Why not retract the rat cancer / GM corn paper?
What did you think those film crews were doing in the lab?
Fallout from “GM food causes cancers in rats” paper
External links
Elsevier Announces Article Retraction from Journal Food and Chemical Toxicology
Study linking GM maize to rat tumours is retracted
Notorious Séralini GMO Cancer Rat Study Retracted, Ugly Legal Battle Looms
Controversial Seralini GMO-rats paper to be retracted
Belated retraction of Seralini’s bad anti-GMO paper
Scientists pledge to boycott Elsevier
Food and Chemical Toxicology Editor-in-Chief, A. Wallace Hayes, Publishes Response to Letters to the Editors
A couple of phrases got used in multiple essays. One was some variation of, “Storytelling is okay for kid’s books, but not for science.” Another was, “Science is supposed to be about the facts.”
Some other researchers have been critical when I say things like this, because they see it as undermining the cause for federal research funding. This is like worrying about the blood loss from a mosquito bite in a patient with gunshot wound through major arteries.
I am indebted to The Science Show for this piece of advice from the late Fred Sanger, who was asked about how to be a successful scientist (my emphasis):
As an instructor, when I’m working with a student individually, I don’t care how prepared a student is. If I can teach a student at an advanced stage, then surely I can back up and teach them the beginner stuff if that person does not know it. I don’t (or shouldn’t) get upset by a student’s ignorance. I should view them as one of today’s lucky ten thousand.
Last year, a paper by Séralini and colleagues came out that claimed eating genetically modified (GM) food caused cancer in rats. The online reaction was swift and sharp: the samples were too small, too many data were not shown, the statistics were odd or missing. The reaction that made it to the journal’s printed pages was almost equally critical.
Doctor Who helped make me become a scientist.
