18 February 2025

How to shape public opinion

 Shana Gadarian just wrote this useful thread on Bluesky. I’ve compiled it with some light editing.

Some brief thoughts from someone who studies public opinion.

The federal firing spree can be made unpopular because it is being done in a haphazard way, there will be huge economic effects not just in DC but also across country and it’s also likely illegal in many cases. But how to make that argument?

First, remember that most people pay attention to politics most of the time and don’t really know how things work so first explain to people how the federal government works. Really! Who allocates money, who gets to spend it, and why is it a problem for an elected person to be firing civil servants.

Much of our policymaking is done in basically invisible ways so it’s not always clear to people what the policies are that affect their lives. (See The Submerged State by Suzanne Mettler.)

Tell people what current policy is and how long it’s been there. People want policy information but if they don't have it, they fill in the details. (See The Invented State by Emily Thorson.) Right now, the current regime is telling them that government is wasteful and inefficient.

Focus on both the immediate effects of federal funding cuts on the health and welfare of specific people and the country as a whole. Tell both sets of stories! What happens to federal workers at NIH when they lose their jobs, their research, and what are the effects on cancer patients.

It is really hard to track things that don’t happen, but also think about and articulate scenarios of what might happen if PepFar doesn’t fund HIV treatments. Babies develop AIDS. Other countries, like China, fill in the gaps of where the US was and win influence in these regions.

Focus on the very unpopular aspects of the cuts. You can’t go to national parks if there are no park rangers and it’s unsafe to go there. National parks are very popular!

People want things to work and they want to be safe. Talk about the direct consequences of things like cutting FAA staff for airline safety. It would be helpful for pilots, flight attendant, and crew unions to be doing this publicly and also privately through lobbying.

Opinion works when it’s collective, so helping people to understand their own values map on to what is happening and how to then translate that into political action. Calls and emails to their members, working with membership groups, letters to the editor, protest. There are many ways to make voices count.

Trace out the economic effects in very local ways. How many people in your local area work for federal agencies. Will farmers stop getting subsidies? Will disabled students stop getting needed services? Which hospitals will have to stop clinical trials? Which people are affected the most?

 


The missing science solidarity

So. In the United States, funding for science has slowed, mass layoffs have cut through federal science agencies like a chainsaw, and someone who doesn’t believe that germs cause disease is leading the country’s health agency.

Are we scientists reacting with anywhere near enough unity to fight back?

Christina Pagel wrote (lightly edited):

Another group who have done little to organise an opposition are scientists and scientific societies.

While there are ongoing lawsuits around pauses in grant funding or pausing international health programmes, these are being driven by labour organizations or non-governmental organizations, not scientists. Instead, some journals, organizations, universities are quietly censoring output of targeted terms like “gender.” Publicly funded bodies such as NASA, NIH and NSF have little choice but to comply. Individual scientists have spoken out, but many are also reporting self-censoring in fear of impact on their careers. 

Possible reasons:

  1. The scientific community is spread across universities, private research institutes, and government agencies and across very different disciplines, making coordinated action – or even agreeing on aligned interests – challenging.
  2. Large national grant making bodies are not independent of the federal government. Scientists often require a funder’s permission to publish and have little formal say in priority areas for funding.
  3. Most scientists just want to work and aren’t “into politics.” Most fields are relatively unaffected.
  4. Many scientists also feel an instinctive aversion to advocacy, worried that it is not scientific or objective enough.
  5. Some who benefit from changes in funding priorities will welcome the opportunity to further their own research.

We saw all these factors play out in the ideological bent to science in Nazi Germany or Stalinist Russia.

The stability of recent decades has left scientists unprepared for when their independence is revealed to be so fragile with few if any legal protections.

Pagels’s list is helpful start, to which I will add.

Point 1: That the scientific community is spread across institutions is one of the reasons we have scientific societies. Academic societies can represent thousands, or tens of thousands, of researchers in the United States and around the world. But the response of scientific societies to the catastrophic events of the last few weeks has been – at best – mixed. 

Scientific societies have the potential to be critical advocates for science in the United States, and organize collective action – by which I mean protests – against the budget cuts and firings and all other forms of federal attacks.

But I think many scientific societies are too cautious, and have spent too much effort worrying about what the most reticent of their members would think. They will need their members to push them – hard – to take stronger actions.

Point 2: Yup. Many scientists are – or were – employees of the American government.

Points 3 and 4: I think these are variations of “Science isn’t political.” We saw this view promulgated late last year in an editorial by the head the National Academies of Science in the journal Science. I didn’t like it then and I think we have seen how badly misguided it was.

Point 5: Thankfully, I have not seen scientists pulling, “I got mine, so what?” I’m sure some exist, but I don’t think there are many.

I will add a sixth point. A few years ago, Randy Olson wrote:

Science is run by committees, top to bottom. Committees don’t lead, they facilitate. They don’t come up with good ideas and make them happen, they wait for individuals to come to them with good ideas that they can support or reject.

I’ve thought about this paragraph a lot. I don’t agree entirely that science is leaderless, but it surely tends to be run by committees. I don’t agree entirely that committees cannot come up with good ideas. But committees are slow and deliberative on purpose, and that is holding us back right now, where we need speed. 

That makes the role of strong existing “committees,” like scientific societies, all the more critical. Let’s not waste time striking new committees. Let’s mobilize the ones we have already.

If we can get them to be strong enough.

Related posts

Okay, stop. Saying “science isn’t political” will not keep science safe from political attacks 

Scientific societies are failing the moment by scrubbing diversity from their websites without explanation 

External links

Christina Pagel on Bluesky

Michael Crighton: The Lost Opportunity 

11 February 2025

These are not serious people

An American politician with a seat in the House of Representatives has introduced a resolution “to acquire Greenland and to rename Greenland as “Red, White, and Blueland”. 

This is real. Not a prank. Not a hoax. Not an imaginary story. Not an April Fools day joke.

External links

H.R.1161 - To authorize the President to enter into negotiations to acquire Greenland and to rename Greenland as "Red, White, and Blueland". 119th Congress (2025-2026)

Carter introduces bill renaming Greenland, February 11, 2025 


#ArsenicLife in soft focus

I am glad to hear that Felisa Wolfe-Simon is doing well and still loves science.

But lots of other information in a new article in the New York Times about Wolfe-Simon is not as warming.

First, the timeline of events circa 2010 described in the article is very fuzzy at best. I’m not sure if it6s wrong or just cloudy.

By focusing on Wolfe-Simon, others involved in this story – notably Rosie Redfield – are reduced to namely “critics.” There are other elements of the story that are maybe over emphasized?

Critique soon became attack, and attack often became personal – focusing, for instance, on Dr. Wolfe-Simon’s appearance, including her dyed hair.

Several articles described her colourful hair positively. And Rosie Redfield dyed her hair, too. I cannot doubt some attacked her appearance, and that any of those comments can hurt bad.

 And soon, Dr. Wolfe-Simon said, she couldn’t get grants or publish papers.

A lot of researchers struggle to get grants or publish papers. There’s no way of knowing what proposals or manuscripts she was submitting or what the comments were.

Second, there is news that the journal Science is considering retracting the original paper. Current editor, Holden Thorp, expands on the possible retraction of the arsenic life paper in a thread on Bluesky. i asked what point a retraction would serve, to which Thorp replied, “Partly because of AI and even before with search engines that only look at the title, papers like this continue to get cited uncritically. ‘Retraction’ is added to the title in these cases.”

This strikes me as reasonable, but I still have some reservations. By the same argument, anyone who only looks at the title and sees “Retraction” will also view that uncritically and not read the retraction notice. This could mean someone would not cite this paper appropriately for its discovery of a new strain of arsenic-tolerant bacteria (which is not contests). 

Retraction is a blunt instrument.

Third, this article paints Wolfe-Simon as someone who was wronged, not as someone who was wrong.

Usually, professional consequences so severe are reserved for those who fabricate data or commit fraud, which no one has alleged with #arseniclife. Why, then, were the repercussions so resounding for Dr. Wolfe-Simon?

And from reading this article, it seems to me that Wolfe-Simon still thinks she found bacteria that use arsenic instead of phosphorus.

She also defended the discovery against scientific consensus. Some see that as an unwillingness to change a conclusion in the face of new information. Dr. Wolfe-Simon would say that information is wrong.

And...

To this day, Dr. Wolfe-Simon defends the work, noting that she wishes the team had saved less data for a second paper. The team published a response to critiques in Science, and Dr. Wolfe-Simon disputes failed replications of their findings. Other co-authors say they also stand by the integrity of the original work.

What I would like to know if whether the co-authors stand by the conclusions of their work. Do they think they discovered a bacterium that uses arsenic in place of phosphorus?

Wolfe-Simon throws Science magazine under the bus a bit:

She wouldn’t pursue a flashy journal that would impose a heavy hand in publication and press, she said.

We absolutely have to make space for making mistakes and being wrong in science. But researchers have to be willing to admit those mistakes.

External links

Her discovery wasn’t alien life, but science has never been the same. (Paywalled.)
 

Subtleties of sex in sticklebacks

Three spined stickleback.
Daniel Bolnick wrote a thread on Bluesky about the biology of sex that I thought would make a great blog post. So I saved him the trouble and compiled it.

This has been lightly edited from the original thread.

Well, no surprise that (Jerry) Coyne used his blog (whyevolutionistrue) to argue with the SSE/SSB/ASN letter about definitions of sex. I’m not going to engage deeply with the letter here. Like any committee-drafted text it reflects compromises; I agree with some phrasing but some could have been clearer.

Instead, I want to respond specifically about his comments regarding my work on stickleback. He points out, quite correctly, that I have personally used the terms male and female, and sex, extensively in my own research on stickleback. True. 

But it misses the point.

Yes, stickleback can generally be categorized as male and female. And they have a XX female / XY male sex determination system (in some species, not in others). And yes, when I run stats I often check for effects of sex, and often find them. I’ve had whole papers devoted to sexual dimorphism. But Jerry might want to talk to me (or another sticklebacker) before making claims about stickleback sex differences. Because its not as clear cut as he likes. 

Yes, the large majority of stickleback I’ve caught are phenotypically male or female, in the sense that they have ovaries or testes. But I have caught quite a few stickleback (hundreds at least) with neither identifiable ovaries nor testes. I have twice seen stickleback with both (yeah, we dissect a lot of fish). These are rare, but they exist. In statistical analyses, these fish get coded as NA, and excluded.  

So when I publish on sexual dimorphism, or use male/female as a factor in my stats (which I do a lot), I am simplifying, setting aside a small minority to focus on overall tendencies of the majority.

That’s okay for stickleback, but its not ethical when applied to humans.

I could readily have published my papers instead with three levels of “sex”: male, female, and “unknown,” with the third group being comparatively small and under-powered for statistical purposes given my sample sizes. 

Then there’s the issue of trait differences between the sexes, which are clear-cut for most individuals in some environments, but less so in other environments. Stickleback are famous for males having bright red chins and blue eyes as a sexual signal... except when they don’t. There are plenty of lakes where I work in BC, in which males are dull brownish-green, and  indistinguishable visually from females. Indeed, in some lakes we have a hard time generating in vitro crosses because we’ll grab a fish we think is male, only to find after we euthanize and dissect it to obtain sperm, that it had ovaries.

For many traits that we have measured (diet, isotopes, morphology, size, color, immune traits, gene expression, metabolomics), males and females are on average different. But for most of these traits the sexes distributions overlap extensively. The magnitude of that dimorphism varies. In some lakes it can be easy to distinguish the sexes (for most individuals), other lakes it can be very hard, even in multivariate trait space. We can measure this dimorphism using things like linear discriminant function analysis (LDA). This LDA can score individuals as to how distinctively male or female they are and it varies numerically in ways we can measure precisely. Some populations are more dimorphic, others less. These differences persist in the lab, meaning there is genetic variation in between-sex differences.

To conclude: yes, I use the terms male and female in my work, reflecting real features of the fish I study. But I also set aside individuals who don’t fit neatly into the categories; again, that's a fair simplification when working with fish, but not morally defensible with people. 

I also emphasize that even after 25 years studying stickleback, I can mis-identify an individual’s sex, even when the gonads are unambiguous. In these cases the gametic sex is a real distinction, but its relation to other traits is complex.

I'm not personally a deep expert on the biology of sex, that's not my specialty. But, if Coyne wants to have a conversation about sex and sexual traits in stickleback, I'd be happy to point out the subtleties. Most of all, the subtlety that seems to elude him is we can recognize male/female differences that characterize many individuals, and use this distinction, yet still acknowledge that these differences do not apply completely to fully 100% of a population.

Anisogamy (Differences in size of gametes - ZF) will surely come up in comments. I don’t have a problem with this criterion as a broad rule, but again there exist exceptions that make it less than universal. Individuals exist that produce no gametes: genetic sterility, environmentally induced sterility, parasitic castration. For instance, the parasite I study, Schistocephalus, can completely sterilize its host in some cases: we find males with no functional testes, females with non-functional ovaries. Do they cease to be males/females because they don’t have any gametes (whether large or small)? 

To wrap up at last: my impression as an empiricist specializing on a particular fish species: stickleback sex is clearly delineated most of the time (hence, I use the term a lot), but not in every single instance. 

The latter is where Jerry misses the point: Once we pivot into the world of humans, the biology of sex in stickleback (and how I simplify it slightly in my work) is irrelevant to the ethical standards by which we treat each other.

Update, 21 February 2025: Nathan Lents provides another great thread on the biology of sex. (Lightly edited.)

Biological sex and sex differences are real, but poorly captured by a simple binary.

No one is saying there is no such thing as “male” and “female.” (Of course there is.)

No one is saying there aren’t difference between “male” and “female.” (Of course there are.)

No one is saying that “male” and “female” aren’t real categories. (Of course they are.)

We’re saying that these categories are continuous. Continuous categories are still categories. There are many examples of continuous categories in biology, but the simplest is developmental age. Child and adult are real categories, with real and important differences. A child and an adult are different and must be treated as such. But there is no hard line we can draw between those two categories. It’s a spectrum, and a gradual transition. They are continuous categories. But that doesn’t mean the categories aren’t real and important.

There are borderline cases where it’s not simple to say which category someone belongs in – child or adult – but that doesn’t mean the categories are meaningless.  All of adolescence could be seen as fuzzy gray area. Child and adult are still different.

Dawkins and Coyne’s “gotcha” moments of finding authors using the term “male” and “female” in their publications isn't the flex you think it is. They (and you!) probably use other continuous categories like adult, juvenile, adolescent. Continuous categories are still categories.

The same is true for speciation, of course. There wasn’t a Homo heidelbergensis woman that gave birth to a H. sapiens baby. (See? I used “woman” and no one will take issue!) The transition between species is gradual and continuous, a spectrum, but species are still different.

And this is what we’re saying with biological sex. The categories are not as easy to define as we thought because almost all sex differences are bimodal, not binary. Except for the gametes (I’ll get to them in a sec), pretty much all sex differences are continuous.

Examples:

Height, muscle mass, hormone levels & ratios, red blood cell count, white blood cell count, basal metabolic rate, bone density, thickness of the corpus collosum, ratio of white matter and gray matter, size of  pre-optic area, breast size, facial hair, waist-hip ratio, libido… These are all features of sexed bodies that show continuous bimodal sex differences. Don’t all of those matter a whole lot more to health, daily life, relationships, than what gamete you make? Yes, gametes are binary, but they also matter very little most of the time.

There are important health differences. Women are nine times more likely to get lupus. Men two and a half times more likely to get Parkinson’s. Women suffer depression more. Men die by suicide more. Women more often anemic. Men more heart attacks. These are important, but have nothing to do with gametes. There are two liver diseases that are, bizarrely, reciprocally bimodal: Women are nine times more likely to get primary biliary cirrhosis cases, while men are nine times more likely to develop primary sclerosing cholangitis.

Recognizing that biological sex is complex doesn’t deny sex differences, it affirms them. I understand why some want to center the gametes – they are simple and binary. But that ignores all kinds of important sex differences all over the body that actually matter. In my view, the fact that people are retreating to gametes to define sex shows that they know that all of the other sex differences are continuous / bimodal.  

Men and women are real categories with real differences, but the categories are continuous and differences are bimodal.

And I haven’t even gotten to intersex people, who are real people that exist. They are almost always healthy, usually fertile, and in any case deserving of dignity and respect. They, too, cannot be ignored or discarded, simply out of preference for nice neat categories.

Biology is messy, adaptable, and endlessly creative. It rarely traffics in binaries and loves wondrous variety. Life will always frustrate the human desire for neat categories with simple, universal definitions. Diversity is a universal feature of life.  Let's acknowledge celebrate that!

Photo by Jason Ching/University of Washington on Flickr. Used under a Creative Commons license.

10 February 2025

Scientific societies are failing the moment by scrubbing diversity from their websites without explanation

I called last week, “The worst week for science ever.” Of course, it was just the worst week so far.

This morning, I saw reports of two large scientific societies – The American Chemical Society (ACS) and the American Geophysical Union (AGU) – editing their websites to remove or reduce mentions of diversity. 

“The ACS has deleted its website on Diversity, Equity, Inclusion & Respect.” – David K. Smith

“The main DEI page from AGU has been modified to take diversity out of the title.” – Celeste Labedz

This is on top of the American Society for Microbiology removing a bunch of pages relating to diversity and replacing them with “Under review.” 

Not all societies are taking this road. The Society for the Study of Evolution released an message today, for instance, said the society still supported “community building in an equitable and globally inclusive manner.” Near the end, it concludes, “we note that the attacks on historically excluded members of society are reprehensible. We encourage our membership to be unwavering in your support of the most vulnerable within the community.”

My hypothesis about these is that somewhere in these big societies, there is one or more federal grants supporting some program or another.  They had lawyers review the executive orders, who suggested that they change the website to avoid getting sued. A lawsuit exposes the entire society to financial risk, or maybe even ruin.

I appreciate that scientific societies want to continue to exist and are sensitive to risk.

But I am deeply disappointed that there seems to be no communication about these changes. Words and phrases and entire pages are disappearing with no warning or notice. That’s not transparency and it sure as anything isn’t leadership.

07 February 2025

Worst week for science ever

Sunday: American Society for Microbiology scrubs mentions of diversity from their websites.

Tuesday: NASA stops everything to scrub mentioned of diversity from their websites; ICE arrests a student on campus; NSF announces plans to make deep cut to staff; USAID is halted.

Wednesday: HHMI kills one of its diversity program; NIH trashes applications for diversity F grants.

Friday: NIH cuts its indirect costs and sets them to 15%.

Yes, these are all American centric. But the United States was, until the start of 2025, the world’s undisputed science superpower. These effects will not be contained within the borders of the United States.

05 February 2025

Untitled post 2025-02-05

The United States has doused itself in gasoline and lit a match.

The only question is whether the fire can be put out with no more than scorched clothes, or whether it will burn until even the bones are ash.

31 January 2025

With funding under assault, time to revisit science crowdfunding

All out attack
In the last two weeks, the new White House has thrown more monkey wrenches into the American scientific research machine than ever before. Grant money has been frozen, and a lot of people who were getting salaries from grants are facing rent day with no way to pay.

For everyone whose grant or pay is in limbo, I’m so sorry. You don’t deserve this bullshit.

There’s a lot to unpack and I won’t try to do it all in this post. But the current crisis did make me think back a few years about science crowdfunding. In particular, I think I had a good point about the importance of diversifying your portfolio for research funding.

So I wrote a little thread on Bluesky suggesting that some scientists might consider what they could do with crowdfunding for research.

This blog post is also a good opportunity to point to this post on Southern Fried Science about Andrew Thaler’s experience with Patreon:

By just about any metric, the return on investment from Patreon exceeds, by an order of magnitude, just about an other funding I’ve received in the last decade. For most of us, we don’t need to move mountains, we just need the space to stop and breathe and create.

I’m under no illusions about what science crowdfunding can and cannot do. And it sucks that I have to bring it up because we are facing an all out attack on research funding. But times are what they are and we have to think in new ways to keep the gears turning.

External links

My Bluesky thread on science crowdfunding

My blog posts about the #SciFund experience

Small drops make mighty oceans: 10 years as a scientist on Patreon

09 January 2025

We were friends once

In 2021, I was in front of the US Consulate in Toronto. And there was a small plaque that was surprisingly moving to me.


As the United States marks the 10th anniversary of the terror attacks of September 11, 2001, in the name of all Americans, Consulate General Toronto thanks the people of Ontario for their support and generosity following the worst attack on American since WWII.

The nearly 3,000 innocent victims from ninety countries included twenty-four Canadians, our friends and family in Ontario, and all of Canada, were with us in our darkest hour, as you always are. Neither terrorism nor any adversity can conquer free peoples, and we are grateful to stand with the best neighbours any nation ever had.

As together we look back, together we go forward.

We are eternally grateful.

September 11, 2011

 

Reading the text of this now is just crushing to me. I’m sad that “eternal” gratitude seems to have lasted less than 15 years since the plaque was revealed.

External links

Thanks and remembrance

08 January 2025

More publishers toying with AI in journal review

The Bookseller is reporting that Springer Nature is testing AI checks in their production pipeline for journal articles.

According to the publisher, the AI systems are designed to alert editors to “quality issues.” I find this frustratingly unclear, much like Elsevier’s testing of production pipeline changes. What, exactly, are they looking for with these systems?

I can see some automatic checking being very valuable. For example, it blows my mind that many journals apparently do no basic checks for plagiarism. I can also see value in an AI system that scanned basic statistical information, like whether reported p values were possible given the sample size, test statistic, and so on. For example, there is a program called Statcheck that has been proposed for exactly such purposes (Nuijten & Wicherts 2023, 2024), although there are ongoing debates about its utility (Schmidt 2017, Böschen 2024).

If the publishers were confident that these systems were genuinely making the peer review process better and were catching things like:

  • Image manipulation
  • Plagiarism
  • Tortured phrases
  • Citation cartels
  • Non-existent authors or reviewers
  • Statistical errors 
  • Undisclosed use of generative AI

All of which are real problems that need addressing. So why are academic publishers being so cagey about what processes they are implementing and what they are supposed to catch? Are they worried that this provides information that cheaters can use to bypass their “quality issues” detection systems? Something else?

Publishers always claim that they add value to academic publication. These new AI checks provide a real opportunity that they could show how they are adding value to academics who are increasingly mad at them and asking “What good are you?”

Related posts

Elsevier turns generative AI loose on manuscripts for no discernable reason

External links

Springer Nature reveals AI-driven tool to ‘automate some editorial quality checks’

References

Böschen I. 2024. statcheck is flawed by design and no valid spell checker for statistical results. arXiv: 2408.07948. https://doi.org/10.48550/arXiv.2408.07948 

Nuijten MB, Wicherts J. 2023. The effectiveness of implementing statcheck in the peer review process to avoid statistical reporting errors. PsyArXiv. https://doi.org/10.31234/osf.io/bxau9

Nuijten MB, Wicherts JM. 2024. Implementing Statcheck during peer review is related to a steep decline in statistical-reporting inconsistencies. Advances in Methods and Practices in Psychological Science 7(2): 25152459241258945. https://doi.org/10.1177/25152459241258945
 
Schmidt T. 2017. Statcheck does not work: All the numbers. Reply to Nuijten et al. (2017). PsyArXiv. https://psyarxiv.com/hr6qy

31 December 2024

Elsevier turns generative AI loose on manuscripts for no discernable reason

From time to time, editorial boards quit journals. It happens often enough that I usually don’t pay much attention. But the outgoing editors of the Journal of Human Evolution described a new complaint, involving – surprise! – generative AI.

In fall of 2023, for example, without consulting or informing the editors, Elsevier initiated the use of AI during production, creating article proofs devoid of capitalization of all proper nouns (e.g., formally recognized epochs, site names, countries, cities, genera, etc.) as well italics for genera and species. These AI changes reversed the accepted versions of papers that had already been properly formatted by the handling editors. This was highly embarrassing for the journal and resolution took six months and was achieved only through the persistent efforts of the editors. AI processing continues to be used and regularly reformats submitted manuscripts to change meaning and formatting and require extensive author and editor oversight during proof stage. 

This is maddening, because it’s yet another example of gen AI creating problems, never solving them.

I am also baffled. Because usually I can at least understand why a publisher has done certain things. Often the explanation tracks back to, “Cut costs.”

But I cannot for the life of me figure out why a publisher would let a generative AI system loose on a completed, edited manuscript. I cannot believe that is in any way cost-saving.

Generative AI is notoriously expensive. And they were doing this to nominally completed manuscripts, added an additional layer of work. If it were in the hopes of saving costs eventually, you would think there would just be some internal testing, not being unleashed like a rabid raccoon on a working journal.

Nor do I think anyone with any publishing experience would believe it would improve the manuscript.

I am left absolutely confused by what Elsevier is thinking here. But moreover, I am am worried that other publishers are going to try the same thing and we just haven’t heard about it yet.

Other major publishers should see an opening here. They could get out and publicly promise academics that the job of final edit stays with human editors.

Unfortunately, I am having a hard time seeing this happening, as I suspect increasingly the heads of these companies see themselves as data and analytics companies more than publishing companies.

Update, 6 January 2025: Retraction Watch has a response from Elsevier to the resignation. They claim they weren’t using generative AI, but admit they “trialled a production workflow” that caused the errors.

I’m not sure that’s much better.

Obviously, you also hope that workflow changes would be well-thought out enough that they wouldn’t introduce mistakes.

But I’m more unimpressed by Elsevier’s lack of transparency. What were they doing to the workflow that caused the mistakes? That the editors thought this was generative AI suggests that Elsevier did not explain the new workflow to the editors well, if at all. 

Update, 10 January 2025: Science magazine is covering this story, and they have quotes from the editors that contradicts Elsevier’s “We weren’t using AI” claim:

According to Taylor and paleoanthropologist Clément Zanolli at the University of Bordeaux, another former editor-in-chief who signed the statement, Elsevier told them in 2023 that AI software had introduced the formatting errors.

Elsevier told Science the same thing they told Retraction Watch: they were testing a new production system. But the statement to Science did not say there was no AI involved, which they said to Retraction Watch.

External links

Evolution journal editors resign en masse to protest Elsevier changes

Evolution journal editors resign en masse 

Elsevier denies AI use in response to evolution journal board resignations 

Journal editors’ mass resignation marks ‘sad day for paleoanthropology’

20 December 2024

Le Monde est à vous: Academic hoaxes in French newspaper article

The second article that arose from my posting of preprints is now available. The title is, “Why scientific hoaxes aren’t always useless.”

Translation of first paragraph by Google Translate:

Canadian biologist Zen Faulkes is not a naturalist, even if he likes to collect crayfish or sand crabs for his research. However, he has a taste for unexpected collections. For several years, he has been collecting, listing and classifying... scientific hoaxes. That is to say, parody, ironic or insane research articles that should never have been published. “Of course, if these texts disappeared, it would not be a great loss. But it is important to keep track of them and try to learn some lessons from their existence,” says the researcher, whose collection spans 432 pages, for forty-two examples.

While I have been very pleased in the last few weeks to have been quoted in a couple of fairly high profile venues, I am struck by the disconnect this week between good professional news and terrible personal news. (There have been a couple of deaths close to our family in the last week.)

External links

Pourquoi les canulars scientifiques ne sont pas toujours inutiles (Paywalled and in French)

Related blog posts

Clearing out the vault by posting preprints

09 December 2024

Bar graphs, how do they work?

I make a brief cameo appearance in a new article about data visualization. Bar graphs are about as simple and basic as you get in data visualization, but a couple of new preprints (one of which is mine) show that people struggle to get even those right. The major preprint by Lin and Landry finds all sorts of issues in bar graphs. Mine is much smaller, and I just want people to label their error bars.

By the way, this was the unexpected call I got after posting a preprint last month.

Related posts

Clearing out the vault by posting preprints

Reference

Heidt A. 2024. Bad bar charts are pervasive across biology. Nature. https://doi.org/10.1038/d41586-024-03996-w

04 December 2024

“Pay me now or pay me later” in reproducibility

“Reproducibility debt” is an interesting and useful take on the matter of reproducibility and replication. I stumbled across a discussion on a recent podcast.

What I like about this phrasing is that a lot of discussion around reproducibility focuses on bad practices. Things like p-hacking, HARKing, and the like. Framing issues around reproducibility as debt makes it more obvious that what we are talking about are trade-offs.

You might have a more reproducible result if you had a bigger sample size and wrote a perfect paper. But that takes time (opportunity costs), and often takes money (financial costs). And there are benefits to getting papers out - both personal (another thing to add to your annual evaluation) and to the community (puts new ideas out and generates leads for others).

In the short term, it can make sense to take on debt. But you will have to pay it back later.

The paper develops a preliminary list of the kinds of trade-offs that cause reproducibility debt. 

  • Data-centric issues (e.g., data storage)
  • Code-centric issues (e.g., code development)
  • Documentation issues (i.e., incomplete or unclear documentation)
  • Tools-centric issues (e.g., software infrastructure)
  • Versioning issues (e.g., code unavailable)
  • Human-centric issues (e.g., lack of funding)
  • Legal issues (e.g., intellectual property conflicts)

It’s very software focused, so I don’t think the list is comprehensive. For example, in biology, reproducibility might become an issue because a species becomes rare or extinct.

If we have reproducibility debt, maybe we can also conceive of reproducibility bankruptcy: a point where the accumulated shortcuts add up to a complete inability to move forward on knowledge.

References

Hassan Z, Treude C, Norrish M, Williams G, Potanin A. 2024. Characterising reproducibility debt in scientific software: A systematic literature review. http://dx.doi.org/10.2139/ssrn.4801433 

Hassan Z, Treude C, Norrish M, Williams G, Potanin A. 2024. Reproducibility debt: Challenges and future pathways. Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering: 462-466. https://doi.org/10.1145/3663529.3663778

External links

To Be Reproducible or Not To Be Reproducible? That is so Not the Question

29 November 2024

A journal paying for reviews

The BMJ will start paying £50 for reviews.

The catch? It’s not for peer review, it’s for reviews from patients and the public. They still expect peer reviews to be done as part of service.

Their announcement post does not describe how they will address several of what I consider to be potential problems. While my previous post was about posting for reviews from academics, I think many of the issues will also apply to public and patient review.

I’m going to sign up as a public reviewer. Because why not? I could use 50 quid. And I’d like to see what this looks like from the inside.

Related posts

Potential problems of paying peer reviewers

External links

The BMJ will remunerate patient and public reviewers

25 November 2024

“Neurosurgery on Saturn” paper shows academics’ blind spots

The planet Saturn.
In the last few days, a bunch of people on Bluesky discovered a paper that has been out for a few months, “Practice of neurosurgery on Saturn.”

Looking through the social media posts about this paper, a lot of people played that favourite academic game, “How did this get published?” Many people suggested it’s a hoax. Academic hoaxes are a particular interest of mine, and I am always looking for the next entry in the Stinging the Predators collection of academic hoaxes.

I didn’t think this was a hoax? Hoaxers usually reveal the prank almost immediately, and this paper had been out for months.

My hunch seems to have been correct. The lead author has an extensive Google Scholar page and said on PubPeer: “it is clear that the document focuses on fictitious and hypothetical situations.” I am not clear about what the point of the article is, but never mind. It’s filed under “Letters to the editor,” which I think is an arena where researchers and journals can be allowed a little leeway.

But this is a good example of something that I think is decidedly lacking in many discussions about academic publishing and academic integrity. In none of the posts I read did anyone do any actual investigation.

Nobody looked to see if the authors were real.

Nobody emailed the authors. It seems to just be happenstance that the lead author saw the PubPeer comments and replied.

Nobody emailed the journal (although editors are often notoriously slow to reply to these sorts of things).

In collecting academic hoaxes, I’ve noticed a similar pattern. People create hoaxes to show that there are bad journals out there that accept anything for money. But by and large, that is where it stops. 

People know predatory journals are out there, but nobody is actively digging behind the scenes to see how they work. How do people decide to start running them? How much money do they make? Why would a scam artist only in it for the money do apparently counterintuitive things like waiving the article processing charges? (There are multiple instances of that in the Stinging the Predators collection.)

A recent paper came out that made a similar point about the lack of investigation about paper mills  (Byrne et al. 2024). 

Academics treat too many of these problems around dubious publishing as some sort of black box that cannot be opened. They only study the outputs. I think someone needs to approach these sorts of problems more like an investigative journalist or an undercover law enforcement officer might.

Go in and find the people who are doing all this dishonest stuff. Get them talking. I want hear what some of the people organizing predatory journals or paper mills or citations rings have to say. Why do they do what they do?

I don’t expect academics themselves to do this. This kind of investigative journalism is expensive and time consuming and being done less and less. But without this kind of insight, we will probably never be able to understand and curb these problems.

References

Byrne JA, Abalkina A, Akinduro-Aje O, Christopher J, Eaton SE, Joshi N, Scheffler U, Wise NH, Wright J. 2024. A call for research to address the threat of paper mills. PLoS Biology 22(11): e3002931. https://doi.org/10.1371/journal.pbio.3002931

Mostofi K, Peyravi M. 2024. Practice of neurosurgery on Saturn. International Journal of Surgery Case Reports 122: 110139. https://doi.org/10.1016/j.ijscr.2024.110139

External links

PubPeer commentary on “Practice of neurosurgery on Saturn”

Altmetric page for “Practice of neurosugery on Saturn”

Google Scholar page for Keyvan Mostofi

Photo by Steve Hill on Flickr, used under a Creative Commons licence. 

Generative AI: Two of of three is bad

There’s an old joke: “Fast, cheap, good: Pick two.”

Generative AI is fast and cheap. It will not be good.

I would like “good” to be a higher priority.

Edit, 1 December 2024: I think academics, under the intense pressure to be productive, prioritize those three characteristics in that order. The first two might switch (grad students are more likely to prioritize “cheap”), but I think “good” is regularly coming last.



22 November 2024

Clearing out the vault by posting preprints

Old books on a shelf
My last jobs didn’t have any expectations for research, so my publication rate slowed significantly. I wanted to do something about that.

A few months ago, I published a submitted but not completed article on paying peer reviewers here on the blog and on Figshare. This got me digging around in folders on my hard drive and I got thinking about other articles that I had written, submitted, but weren’t accepted. And at the time, I just ran out of steam to revise and resubmit them in ways that would satisfy an editor and a couple of reviewers. 

So in recent weeks, I have taken to submitting some of those manuscripts to a preprint server.

In the past, I was a little cool to the value of preprints. I always thought they had some uses, but I was skeptical that they would replace journals, which was the stated wish for some of the strongest preprint advocates. I was always worried about the Matthew effect: that famous labs would benefit from preprints and everyone else would just have more work to do.

But since I last wrote about preprints, the attention landscape has changed. More people in biology have started scanning through preprint servers part of their routine. I was surprised when a reporter emailed me about one of these preprints and wanted to chat for an interview. I don’t think that would have happened a few years ago.

Whether these will be “final” public version of these little projects, I can’t say. I have other projects that I want to get out that have not been written up at all yet that I want to try to get into a journal. But I am glad that every one of these received at least a little attention on Bluesky.

Here’s the list of my recent preprints, all on bioRxiv.

Update, 26 November 2024: I’ve now had two journalists reach out to me because of these preprints. I’ve never had that high a level of interest from the dozens of peer-reviewed journal articles I published.

Related posts

A pre-print experiment: will anyone notice?

A pre-print experiment, continued  

A pre-print experiment, part 3: Someone did notice  

Photo by umjadedoan on Flickr. Used under a Creative Commons license.

18 November 2024

The cat that fooled Google Scholar, the newest hoax in my collection

I finally got a chance to update my collection of academic hoaxes!

Stinging the Predators 23.0: Now with a cat! (The Internet loves cats, right?) http://bit.ly/StingPred

I am now up to 42 academic hoaxes, which is triple the number that I started with in version 1.0 in 2017.

The latest hoax targets not a predatory journal, but an academic search engine. While this is unusual, it is not the first hoax that was pulled to show how easy it is to manipulate Google Scholar. (One of the things that has been interesting to me as this project has continued is that many hoaxers feel compelled to make the same point again.)

And one other things that has been rewarding is that this collection, which I’ve only ever had on Figshare and promoted on my socials and personal websites, has been viewed tens of thousands of times and has been cited by scholars writing in proper journals a few time.

External links

Faulkes Z. 2024. Stinging the Predators: A Collection of Papers That Should Never Have Been Published (version 23). Figshare. https://doi.org/10.6084/m9.figshare.5248264