Should a scientific paper be retracted because it is mistaken?We’re not talking here about misconduct, or deliberate fraud. We’re talking about a result that is, for whatever reason, wrong: a false positive or a miss, or an overly enthusiastic interpretation, or a good old honest mistake.
At the Retraction Watch blog, Tom DeCoursey argues that papers that are wrong should be retracted from the scientific record. His main argument is that people waste a lot of time trying to reproduce results that later papers have been unable to confirm.
This may be a rather different view of retraction than has typically existed. My impression is that previously, retraction occurred primarily when there was scientific misconduct: fabricated data, or an editor doing an end run around the peer review process. In the medical literature, retracting could also occur in case of an error that might kill people from mistreatment (“lethal error”; Horton 1995).
I get the impression that papers are getting retraction for a much wider range of reasons than ever before, for reasons that have nothing to do with the quality of the science (e.g., authorship squabbles; embargo violations; pressure from bloggers).
I went looking for whether anyone has conducted research on the reasons for retraction. A quick search turned up work by Snodgrass and Pfeifer (1982). Of the papers they looked at, 94% of the retractions they studied had a reason given, but they don’t break down those reasons into any categories.
Six percent of retractions were not explained at all. I’ve seen papers much more recently with no reason given, so the practice hasn’t stopped.
If we widen the use of retractions, all sort of questions are raised. What level of evidence, of failure to replicate, should be enough to warrant a retraction? One of my own recent papers was an extended attempt to replicate another experiment, without success. But it would be presumptuous of me to demand the original paper be retracted.
In the case of honest mistakes, how should a retracted paper factor into promotion, tenure, or funding decisions? And how should journals seek to notify readers? Remove the paper from the record, which is how retraction was supposed to work? Label it as retracted?
Another take on this from Biochem Belle.
Additional: An abstract describes a study (which I don’t think has been published yet) that ranks reasons for retraction: Misconduct makes up 15% (data fabrication, 5%; data falsification 4%; plagiarism, 16%); “mistakes” make up a larger component of about 29% (honest research errors; 28%, non-replicable findings, 11%). Then you have issues that are harder to categorize: redundant publication, 17%; disputed authorship/data ownership, 5%; inaccurate/ misleading reporting, 4% (not sure what that could be); and the kicker, no stated reason at 9%.
Reference
Horton R. 1995. Revising the research record The Lancet 346(8990): 1610-1611. DOI: 10.1016/S0140-6736(95)91935-X
Snodgrass GL, Pfeifer MP. 1982. The characteristics of medical retraction notices Bull Med Libr Assoc 80(4): 328-334.
I guess it depends what you mean by wrong, right? If its wrong in the sense that the lab originally publishing the paper cannot reproduce its own results, then maybe a retraction is in order. If its wrong in the sense that a different lab proved that the original hypothesis was wrong because they did not do the correct experiments, etc. and then provide a "correct" answer, then obviously it shouldn't be retracted, its part of the back and forth that is the process of science.
ReplyDeleteWhere it gets tricky is when one lab cannot reproduce another lab's results. Then the question is, who is right? Assuming the original lab's results were not fabricated, then it's tricky to say why one lab got a specific result and why the other didn't. Over time, a consensus may emerge that shows that many other people couldn't reproduce a result and the result will fall out of favor, but I don't think it should be eliminated from the literature.
If there is no misconduct and were no apparent mistakes in the experiments done, the paper should stay around, regardless of who replicates it (or fails to replicate it, even if it is the original lab.) Experimental papers are not explanations of what will be found in the future; they are records of what did happen under certain circumstances in a specific experiment or set of experiments. "Failure to replicate", unless it involves misconduct or a mistake (I can think of a famous retraction where methamphetamine was mistaken used when the experimenters thought they were using MDMA,) isn't grounds for a retraction, because the resulting conflicting reports still serve as a useful scientific record of what people have done and how it turned out. At the very least, they tell us that there are other factors at work besides those that the experimenters explicitly controlled for or manipulated.
ReplyDelete