Discussion around the report of neutrinos travelling faster than light continue apace. Everyone is talking about it in the scientific community and most media outlets have also covered it.But one thing that has not been often commented upon is that this finding hasn’t been peer-reviewed. It’s a pre-print in ArXiv.
You won’t see the lack of peer review mentioned in Nature News, ScienceNow!, New Scientist, Uncertain Principles, Starts With a Bang!, or Victor Stenger in The Huffington Post, to name some of the commentary. Some of those do mention that the paper is deposited on ArXiv, but not many will understand the role that ArXiv plays in the physics community.
One of the first questions I try to get my students to ask in evaluating claims is, “Was that finding in a peer-reviewed journal?” I try to instill more sophisticated evaluation strategies as we go on, but that question alone can sort out a lot of rubbish in a first pass.
But in this case, nobody seems to care. This made me ask a lot of questions.
Does it mean that peer review is adding no value for the physics community? Everyone is talking about it and trying to think of ways to explain the result and replicate it. Nobody is waiting for the peer-reviewed paper to come out. Is there any incentive for these authors to submit this manuscript to a peer reviewed journal?
Does it mean that this group of scientists built up an unusually sold track record for careful and meticulous work? As an outsider to the field, I have no way of assessing that. Is this why people don’t care that this is not peer reviewed?
Does this mean that there something special about the high-energy particle physics community that makes the reporting practices different than other fields? Would a similarly contrary claim in other fields be treated as openly and as with as much enthusiasm in the scientific press? I’m having hard time thinking of a claim in biology that could be as contradictory as the notion that something could travel faster than light. Maybe someone claiming that DNA was not the hereditary material in cells would be on that level.
If someone in another field tried to make a similarly big claim outside a peer-reviewed journal, would they be laughed out of the room? Would journalists be tougher on them?
The sense that I've gotten is the physics community is more 'open science'-based than other fields (especially biomed fields), to some extent because they can be (specialized approaches/instruments few others have) and need to be (specialized approaches/instruments few others have ;) ). Although no one has mentioned the lack of peer-review, I do think it's important that (at least in articles I've read) that the investigators have posed it as, "Hey-this is completely unexpected. Tell us we're wrong. Provide alternative explanations. Repeat the experiments if you can." It lends the impression that this is about finding the answer, not selling their story.
ReplyDeleteI don't have a ready link to the news story I read, but according to that they were going to defer publication until groups at other sites had replicated (or not) their results. I would assume (but you know what that does) that the paper in ArXiv is there to help with the replication process.
ReplyDelete