Reporting on data that has not gone through the peer review process as if it were truth is not responsible journalism.
I have also been shocked, shocked, I say, to see a paper deposited in arXiv being reported around the world by researchers and journalists alike. Nobody commented that it hadn't been accepted in a peer-reviewed journal.
Except I'm not talking about Rosie Redfield’s arsenic life paper.
I'm talking about the reports of faster than light neutrinos from OPERA.
Did we hear howls of outrage from the physics community over the coverage of the story? More like chirping crickets. Physicists were right there in the thick of the discussion.
This is an example of the differing cultures of the fields. Physics has developed a pre-print culture where people stake their claims with manuscripts. Biology has developed a culture where people stake their claims with final publications. But cultures change, and it’s individual cases like this one that provide a lot of the push to change.
Just because biologists normally only make findings public very near the last step in the publication change does not change that anything made public at any stage is fair game for reporting.
Dr. Redfield made her work public earlier than others would have done. Unusual, but I cannot see the ethical issue with reporting on information that she has voluntarily shared. Indeed, if her work does not pass peer review, the reporting that is going on right now can add context to that story.
For that matter, journalists have covered results presented at scientific conferences for decades. I have never heard serious suggestions that conference reporting is unethical.
The arsenic life story itself tells us that because a paper has been peer reviewed does not make it automatically credible to other researchers. Biologists on the whole weren't convinced by the claim.
One quick thing to point out is that abstracts submitted to conferences are usually peer-reviewed (at least in my specialty), but not as intensely figure-driven as journals. Data (even p-values) are usually included in the abstracts.
ReplyDeleteAs a side note, I've often seen reports from meetings often frame findings as being preliminary, but this may be discipline-specific.
I think the big problem is when partial stories are framed as complete. Omitting the status of the data source (published, under review, from a meeting) is critical information that needs to be reported and may even be more important than any supposed "findings."
I believe peer reviewing conference presentation abstracts is common in engineering.
ReplyDeleteIn contrast, I know of no conference in biology that peer reviews abstracts.
I'll grumble if I see sensational reports that don't mention, "This is a conference presentation." But that's an issue with the level of care in the reporting, rather than ethical issue of reporting on something that isn't peer reviewed.
Pretty much everything medical (especially society-related) that I know of is reviewed, but I can't use specifics here...
ReplyDeleteBut that's an issue with the level of care in the reporting, rather than ethical issue of reporting on something that isn't peer reviewed.
ReplyDeleteExactly! Careless, sensationalist reporting is bad, no matter what the status of the paper supposedly being reported on.
Did anyone bother to read Redfield's paper? She couldn't replicate the growth rate reported by FWS, yet still went ahead with the arsenic assays to determine if it was incorporated. The rebuttal for the Redfield work is going to be, "Well, they couldn't get the cells to grow correctly, so that's why they couldn't replicate our findings." Reporting on this paper as if it has killed arsenic life, which is what numerous outlets have done, really isn't responsible. It's going to take more work from more labs or a retraction from FWS to finally nail this story. I'm not saying the results of the original paper are right, or that peer review is some holy grail, but reporting on a negative result this early as some kind of perfect validation is not what science reporting should be about. I agree that science and peer review is a process and it should be reported on in a similar manner. I'd be much happier if the stories guarded these initial results with something like, "First results hint that arsenic life is not real, further work needed."
ReplyDeleteI'm not sure that FTLN illustrate your argument very well. That's a perfect example of another story that NEVER should have been reported on in MSM until researchers were able to validate the results. Peer review would have saved an incorrect story from hitting the airwaves. Which headline are readers going to remember? The one on the front page in big bold print that says "NEUTRINOS TRAVEL FASTER THAN LIGHT" or the rebuttal a week later on page 12 next to the obituaries that reads "Mathematical error, neutrinos not super"
Did anyone bother to read Redfield's paper? She couldn't replicate the growth rate reported by FWS, yet still went ahead with the arsenic assays to determine if it was incorporated.
ReplyDeleteI did read it. You're wrong, she could get the growth rate to replicate. She had to add a little phosphate to get it, but she replicated it.
Did the phosphate concentration replicate that contaminating the FWS paper? We don't know, because they didn't report that (or, probably, test it). They were sloppy; Redfield was not.
You seem to be arguing that sloppy, careless science can stand forever, because it's not reproducible. Is that really what you want to say?
It's not that Redfield failed to replicate FWS. It's that FWS generated non-reproducible work. Non-reproducible work is bad science, and shouldn't get a pass because it's bad science.
(If you're just saying that FWS will blow it off and claim that it doesn't count, I'm sure you're right; and I'm sure her claims will be ignored by the vast majority of scientists, because her non-reproducible work doesn't stand up.)
More to the point, the physical chemistry is reasonably well-understood. DNA with an arsenodiester backbone is not chemically plausible. The half-life of an arsenoester bond in water at 25° C is 60 milliseconds. That's for one bond.
ReplyDeleteThe half-life of a phosphoester bond -- used as the DNA backbone in all known organisms -- is about 30,000,000 YEARS under the same conditions.
http://pubs.acs.org/doi/pdf/10.1021/cb2000023
An back-of-envelope calculation: the E. coli genome has about 10 million phosphodiesters in the two backbones of its ~5 Mbp chromosome. If they were all arsenodiesters, a new break would appear in the DNA backbone roughly once every 10 nanoseconds. Something like 6 trillion backbone breaks a minute.
Arsenic? You're going to need a bigger DNA repair system.