What the Coburn report has in common with arsenic life

Last week saw two big science stories hit that at first glance are unrelated. But I think they are symptomatic of the same underlying attitudes and problems.

First was Senator Coburn’s report on the National Science Foundation, which I covered here.

In response, the Los Angeles Time ran this excellent article about “Duh science.” It talks about why people often resist “what everyone knows.”

(C)onsider the case of Harvard sleep expert Dr. Charles Czeisler, who has spent about $3 million over the years demonstrating over and over that doctors who don’t get enough sleep make mistakes on the job. ...

Everyone had an anecdote. Czeisler had data. “It was dismissed out of hand,” he said. “They use the same argument over and over, even when we”ve tested it. It drives me up the wall.”

The researcher who built the treadmill for the shrimp, David Scholnick, explains (defends) his research in this article in The Toronto Star.

Second was the publication of a series of technical comments on the “arsenic life” paper by Wolfe-Simon and colleagues (editor’s comment here). Lots of commentary has emerged, again, but I was particularly struck by Ericka Check Hayden’s article in Nature, where several people bluntly say that they don’t want to replicate the work.

(M)ost labs are too busy with their own work to spend time replicating work that they feel is fundamentally flawed, and it’s not likely to be published in high-impact journals. So principal investigators are reluctant to spend their resources, and their students’ time, replicating the work.

“If you extended the results to show there is no detectable arsenic, where could you publish that?” said Simon Silver of the University of Illinois at Chicago, who critiqued the work in FEMS Microbiology Letters in January and on 24 May at the annual meeting of the American Society for Microbiology in New Orleans. “How could the young person who was asked to do that work ever get a job?” Silver said.

I think there’s something unstated in Silver’s quote. He’s probably asking, “How could someone doing that work get a tenure-track job in a major research university?” Not everyone aspires to that goal.

Refuting another scientist's work also takes time that scientists could be spending on their own research. For instance, Helmann says he is in the process of installing a highly sensitive mass spectrometry machine capable of measuring very small amounts of elements. But, he says, “I’ve got my own science to do.”

This is bad news for science all round. This entry by PZ Myers beat me to it:

I'm suggesting that they are symptoms of something rotten in the world of science. Testing claims ought to be what we do. If the journals are going to fill up with positive claims thanks to the file-drawer effect, and if nobody ever wants to evaluate those claims, and if negative results are unpublishable, the literature is going to decline in utility for lack of rigor and evaluation.

And even if researchers are willing to do the replication, journal editors don’t seem to see this as important. An excellent recent example detailed by Ben Goldacre was the publication of findings that seemed to suggest precognition. The author, Daryl Bem, correctly realized this was an extraordinary claim, and in his paper, stressed the importance of other labs trying to repeat the finding.

The journal wouldn’t publish the papers. The journal seemed to have a blanket policy (informal or not, I don’t know) not to publish replications.

I’ll add this. Of all the papers I’ve published so far, by far the hardest one to get into print was a replication.

The common link to these two stories?

Both are about the tension between wanting breakthroughs and the reality that science usually progresses in slow, hard fought, millimeter by millimeter increments.

Politicians wants breakthroughs because they see anything else as a waste of taxpayer money. Consequently, it’s easy to look in and see a single research project as stupid because you have no context.

Scientists want breakthroughs because discoveries can make careers. It’s no accident that the arsenic tolerant bacteria’s name is an acronym for “Give Felisa A Job.”

Lots of people (including editors) overwhelmingly want the breakthrough, the identifiable “Eureka!” moments. We need tell more stories of scientific progress that unfolds over years and decades, which is a great opportunity for bloggers.

Related posts

Tales to astonish
Original and transformative
My posts on arsenic life here, here and here.

Photo by quarksteilchen on Flickr; used under a Creative Commons license.