Big blue sky

 

Bluesky is become my main social media platform now.

I mean, they called me an “elder”! Not just in an, “Wow, you old” way. In a nice way!

Yes, Threads has picked up, and I still poke around Mastodon sometimes, but Bluesky has left me feeling... nice? As of right now, it probably has more of the online science crowd. The developers seem to be making decent decisions. And it’s not overrun with ads and company accounts. I’m sure those last will come, eventually, but not yet.

I will probably start to occasionally pull from Bluesky and write about it in the way as I have done before, but I don’t want to add another tag to my blog. I will be using “Twitter” as the tag for my social media posts here on the blog. Because from my current point of view, that tag is more about the microblogging format now than about the specific platform.

P.S.— Blusky game now is to see who’s enrollment number is a prime. Mine isn’t, but 265,499 is 13 squared times 1,571. Pretty good.

Scholarly publishers sued

Complaining about academic publishing is somewhere between “hobby” and “righteous cause” for many researchers. So I imagine that many will be cheering loudly at the news that six major academic publishers are getting sued.

People are likely to focus on the three things the case says makes for unfair business practices:

1. That peer reviewers aren’t paid, and that this is enforced by a “tit for tat” system where journals won’t publish your paper if you don’t review. (I know zero well documented cases of this.)
   
2. That journals will not consider work submitted or published elsewhere (the Ingelfinger rule, I think).
3. Submitted papers can’t be shared while under review.

If I wanted to sue academic publishers, I’m not sure these are the lines of attack I would use. These claims seem hard to stick to the publishers.

The first two points are about practices go back decades, well before the consolidation of so much academic publishing into a few companies. Publishers can say, “These were practices established by the community that we adopted.”

And there are many journals not run by these companies that do the same things. I suspect that the “We don’t consider work under review elsewhere” is common across all publishing, not just academic journals. Publishers can say, “If all these other publishers have these practices, we are just in line with industry standards. And by the way, why are we being selectively prosecuted?”

The third point, that journals “prohibit scholars from sharing advancements in submitted manuscripts while those under peer review” seems to pretend that preprint servers don’t exist.

But these counterexamples are beside the point, because legal question here isn’t whether journals do these things or that they are bad for research. The legal question is whether the publishers conspired to create those conditions.

I think that will be hard to show.

Now, may the plaintiffs can produce something like internal memos or emails between the publishers trying to kill proposals to pay peer reviewers. The academic equivalent to the tobacco industry’s “Doubt is our product” memo. That would be truly devastating. And, in all honesty, I wouldn’t put it past publishers to have some of these emails buried on servers someplace. 

This lawsuit will be interesting to watch. Maybe the plaintiffs aren’t expecting to win, but are doing some consciousness raising. Even if they lose, this lawsuit might do some good by getting academics talking about publication, and maybe by prodding publishers to do better work.

External links

Academic Journal Publishers Antitrust Litigation

Prodding the behemoth with a stick 

Banzai!

Yes, this post is a not too subtle acknowledgement of the fact that I be old, but I don’t care. Buckaroo Banzai is one of those movies that just shaped such a huge part of my mental furniture that I have to acknowledge that today is the 40th anniversary.

The attraction for me is the lead character. I wanted to grow up to be Buckaroo Banzai. Still do. I always loved the portrayal of someone smart and a willing to go full bore into pursing passions: science, music, and helping others. And maybe that I also had a name that evoked Japan increased my identification with the character a bit.

Just before he utters one of the film’s most famous lines, “No matter where you go...”, there’s another moment that underscores the character:

“Don’t be mean. We don’t have to be mean.”

It reminds me a lot of a line that is shared often around on acadamic social media: Lots of people are smart. Distinguish yourself by being kind.

Dissertation thank yous

There is a lot of jaded-ness in academia right now. 

But there is still a lot to celebrate and a lot of sense of achievement. Particularly for graduate students.

So I just love this look at doctoral dissertation acknowledgements. I was particularly struck by this reflection:

No matter how impenetrable the thesis title, the project’s success always seems to come down to the same simple thing: other people.

While graduate programs tend to focus on students, it’s a great reminder that humans are a social species and we are deeply affected by those around us.

External links

The unexpected poetry of PhD acknowledgements

You need how many letters to prove you should get tenure?!

Many academics badly underestimate how much diversity there is in how universities do business. This includes me. Because I was caught flat-footed by a new paper that examines a common part of the tenure process for academics.

When someone is going up for tenure and promotion, it’s common to ask people from outside the university to write a letter describing how this person fits into the research community. These external letters can be a good safety valve to prevent a department from glossing over problems with one of their faculty.

When I was at UTRGV, the “external letter” requirement was just getting implemented. One of the key factors was how many letters to ask for. Speaking at a panel this week in DC, someone mentioned the practice, and talked about the difficulty in getting “three letters.”

Three letters turns out to be on the low end of the task. The most common minimum number of letters was five.

Some universities require a minimum – I say again, a minimum – of ten external letters.

All I can think of is, “How insecure do you have to be in your decision to hire someone that you need external validation from ten other people?”

References 

Hannon L, Bergey M. Policy variation in the external evaluation of research for tenure at U.S. universities. Research Evaluation: rvae018. https://doi.org/10.1093/reseval/rvae018

First, some backstory.

I wrote this essay for a journal. It was provisionally accepted, but it needed revision. Then I changed jobs, and I was never able to sit down and get the revisions done. A few other things I observed (unrelated to this article) made me reluctant to resubmit this to the journal I originally sent it to. But it’s hard to find a journal, or even a preprint server, interested in opinion articles like this. So I didn’t revise and resubmit it.

But because this topic makes the rounds in academic social media routinely, I thought this was worth sharing in some form, even this first unrevised version.

Because this has been sitting on my hard drive for a while, the article has, to some degree, been overtaken by events. In particular, my comments about using AI for peer review were seen as unrealistic when I submitted this – but that was before the release of ChatGPT.

This article is archived on Figshare: https://doi.org/10.6084/m9.figshare.25746996

 

---

 

Potential problems of paying peer reviewers

Zen Faulkes

School of Interdisciplinary Science, McMaster University (Note, 3 May 2024: I am no longer affiliated with McMaster. Please don’t bug them.)

Pre-publication peer review organized by academic journal editors, funding agencies, and so on (simply called “peer review” from here on) is viewed as a hallmark of quality academic publication. But there are concerns that “reviewer fatigue” is causing trouble for peer review. Authors complain that they receive too many requests to review manuscripts, and editors complain about the difficulty of finding willing and qualified peer reviewers (Fox 2017, Goldman 2015, Vesper 2018). Therefore, many academics suggest that publishers should pay researchers for peer review. This idea appears popular: a poll asking, “Do you think reviewers should be paid for reviewing?” found 85.6% of respondents voted yes, compared to 11.3% voting no, and the remaining 3.0% unsure (n=1,234, with 6.2% opting “Show results” instead of voting) (Academic Chatter 2022).

It is easy to see the appeal of paid peer review. It provides a new incentive for people to agree to review. Reading papers thoroughly and then crafting coherent, useful reviews of the manuscript is work. It takes time, effort, and has opportunity costs for the reviewer. Professional work deserves to be compensated, and many are galled that they are expected to “work for free.” The argument is that publishers are exploiting academics who are foolish enough to provide their work for free. The feeling of exploitation is made worse because most academic publishers are for profit businesses that are extremely profitable. Academics often describe publishers’ profits as “obscene” (Stafford and Brand 2021, Taylor 2012). Even non-profit publishers are criticized for the salaries of those working for the company (Eisen 2016).

It is understandable that academics would resent businesses that make money from scholarly goods and services, which are often publicly funded, when those academic are struggling financially. Compensation for postdoctoral researchers and contingent faculty jobs can be below the poverty line (Gee 2017, Semeniuk 2022). One UK funding agency sent an email suggesting doctoral students could work extra jobs to make ends meet, and listed examples such as “Babysitting” and “Fruit picking” (Lowe 2022). Money is a problem for many academics. But when so many problems can be solved with money, it is easy to forget that not all problems can be solved with money. 

Money cannot buy time

Major main reason for prospective reviewers declining invitations is lack of time (Tite and Schroter 2007). When time is at issue, paying for reviews is unlikely to get people to accept the review (Tite and Schroter 2007).

Some aspects of peer review might be improved by paying reviewers, but the overall process of scholarly communication might not be improved.

Payment suggests corruption

Researchers have a long tradition of providing academic knowledge, and other goods and services, for minimal to no costs: “We don’t do it for the money.” Because of this, science was mainly practiced by the wealthy before it was professionalized in the 20th century. This was seen as a way of ensuring research integrity: wealthy researchers were seen as honest, because they would have no financial incentive to lie about their findings.

Unsurprisingly, one of the most common tactics of anti-science campaigns and conspiracy theorists is to argue that scientists are corrupted by money. “Shills for big pharma” (Blaskiewicz 2013, Smith 2017), “getting rich on grant money” (Merzdorf et al. 2019) “companies stifle world changing technology / medical treatments because it would cut into profits” (Demeo 1996), and exhortations to “Follow the money!” are but a few of the variations of this. While it is always difficult to measure the persuasiveness of arguments “in the wild,” that these arguments are used so often over so many years suggests people truly are persuaded by them.

If reviewers were paid, anti-science campaigners could easily point to reviewer payments as evidence that academic publishing is merely a money-making game for insiders (researchers).

Unqualified reviewers

Peer review is valued because it is conducted by fellow experts in a field. Because there are costs to reviewing and little to no recognition, there are few incentives for reviewers to review a paper that is not professionally relevant to them.

Paying for reviews creates a generalized incentive for reviewers to say yes to any paper they are invited to review, regardless of their knowledge of the topic. Because reviewers’ identities are typically confidential, authors who receive poor reviews may want to argue that they had a reviewer who only wanted the money.

Unhelpful reviewers

The overall quality of reviews varies. Some reviewers are detailed and contain multiple specific actionable items that can improve the paper. Other reviews are cursory and contain only vague statements that cannot be easily addressed by authors. Other reviews may be substantive but make unreasonable requests for new experiments and analyses. 

Worse, there are many examples of egregiously bad reviews, running the gamut from unhelpful to rude, racist, and sexist commentary unrelated to the content of the article (Silbiger and Stubler 2019, Woolson 2015).

Fair compensation for review

Whether a journal should pay for a perfunctory or insulting review is a specific example of a much larger issue: determining what is fair compensation for a review. Academic manuscripts differ wildly in length and complexity. The amount of effort that people must put into writing reviews also varies. An experienced researcher may be able to assess and review a manuscript very quickly and respond with a concise but informative review. 

Because so much discontent about volunteer peer review concerns academic work being undervalued, those arguing that peer reviewers should be paid should also make concrete suggestions for pay scales.

Alternatives to reviewer payment

Paid peer review aims to solve two problems: unwillingness to review (“reviewer fatigue”) and uncompensated labour. A more radical solution to these problems would be to take humans out of the peer review process. There are at least two ways this could be done.

The first solution would be hand over the bulk of peer review to specialist artificial intelligence (AI) expert systems that can review manuscripts. Recent advances in AI systems that can process and manipulate natural language texts, such as GTP-3, this may be viable in the near future (Schulz et al. 2022).

The second solution – less speculative but more radical – is to do away with peer review entirely (in the limited sense of prepublication review organized by editors). There are strong criticisms that pre-publication peer review does not provide the quality control and improvement that researchers want (Smith 2010). Ongoing post-publication peer review might provide an alternative to the journal centered form of peer review that is currently the norm.

If neither of these suggestions are embraced by the research community in the near future (which seems likely), it may be possible to address issues with voluntary peer review by more careful accounting of service obligation.

The main components of an academic’s job are often described as research, teaching, and service. Academics in universities are often explicitly told how much of their time should be allocated to each of those three parts of the job, but service is often the smallest piece of that budget. It is unsurprising that peer review, a service commitment to the research community, suffers when institutions explicitly show that service is the smallest part of the job description. Nevertheless, researchers with service obligations might be able to account for time spent on peer reviews more carefully to argue for release of other service commitments (e.g., committee work). Similarly, administrators might work to ensure that service obligations are not overlooked or satisfied by trivial “box checking” services.

Service loads in peer reviewing are also distributed unevenly. Journal editors often ask people to review whose jobs have no expectation of service. This includes early career researchers such as grad students and post docs, researchers who hold jobs outside academia, and retired academics. This is hardly fair to those individuals, and people in these positions have the best case for being paid for review because it is literally not their job.

Journal editors might also be able to better balance the service load by broadening the pool of who is asked to review. Many who are willing to provide peer review are rarely, if ever, asked to do so (Vesper 2018). In particular, researchers in “emerging economies” are probably underused as potential peer reviewers (Vesper 2018).

References

Academic Chatter. 2022. Twitter. https://twitter.com/academicchatter/status/1543229417060696066

Blaskiewicz R. 2013. The Big Pharma conspiracy theory. Medical Writing 22(4): 259-261. https://doi.org/10.1179/2047480613Z.000000000142

Demeo S. 1996. The corporate suppression of inventions, conspiracy theories, and an ambivalent American dream. Science as Culture 6(2): 194-219. https://doi.org/10.1080/09505439609526464

Eisen M. 2016. On pastrami and the business of PLOS. https://www.michaeleisen.org/blog/?p=1883

Fox CW. 2017. Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution. Scientometrics 113(1): 465-477. https://doi.org/10.1007/s11192-017-2489-5

Gee A. 2017. Facing poverty, academics turn to sex work and sleeping in cars. https://www.theguardian.com/us-news/2017/sep/28/adjunct-professors-homeless-sex-work-academia-poverty

Goldman HV. 2015. More delays in peer review: Finding reviewers willing to contribute. https://www.editage.com/insights/more-delays-in-peer-review-finding-reviewers-willing-to-contribute

Lowe A. 2022. Twitter. https://twitter.com/adriana_lowe/status/1549754463619108865

Merzdorf J, Pfeiffer LJ, & Forbes B. 2019. Heated discussion: Strategies for communicating climate change in a polarized era. Journal of Applied Communications 103: 3. link.gale.com/apps/doc/A600269487/AONE?u=anon~a35eccce&sid=bookmark-AONE&xid=f37daf5b

Schulz R, Barnett A, Bernard R, Brown NJL, Byrne JA, Eckmann P, Gazda MA, Kilicoglu H, Prager EM, Salholz-Hillel M, ter Riet G, Vines T, Vorland CJ, Zhuang H, Bandrowski A, & Weissgerber TL. 2022. Is the future of peer review automated? BMC Research Notes 15(1): 203. https://doi.org/10.1186/s13104-022-06080-6

Semeniuk I. 2022. Prominent researchers urge Ottawa to increase top science scholarships above poverty line. https://www.theglobeandmail.com/canada/article-prominent-researchers-urge-ottawa-to-increase-scholarships-for-top/

Silbiger NJ & Stubler AD. 2019. Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ 7: e8247. https://doi.org/10.7717/peerj.8247

Smith R. 2010. Classical peer review: an empty gun. Breast Cancer Research 12(Suppl 4): S13. https://doi.org/10.1186/bcr2742

Smith TC. 2017. Vaccine rejection and hesitancy: A review and call to action. Open Forum Infectious Diseases 4(3): ofx146. https://doi.org/10.1093/ofid/ofx146

Stafford T & Brand CO. 2021. Commercial involvement in academic publishing is key to research reliability and should face greater public scrutiny. https://doi.org/10.31222/osf.io/rjmvh

Taylor M. 2012. Academic publishers have become the enemies of science. https://www.theguardian.com/science/2012/jan/16/academic-publishers-enemies-science

Tite L & Schroter S. 2007. Why do peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health 61(1): 9-12. https://jech.bmj.com/content/jech/61/1/9.full.pdf

Vesper I. 2018. Peer reviewers unmasked: largest global survey reveals trends. Nature. https://doi.org/10.1038/d41586-018-06602-y

Woolson C. 2015. Sexist review causes Twitter storm. Nature 521: 9. https://doi.org/10.1038/521009f

Open access: What is a paper for, anyway?

Brian McGill at Dynamic Ecology blog has an interesting overview of publishing trends. The paragraph that seems to have gotten the most traction is this one: 

Open access has been a disaster. Scientists never really wanted it. We have ended up here for two reasons. First, pipe dreaming academics who believed in the mirage of “Diamond OA” (nobody pays and it is free to publish). Guess what – publishing a paper costs money – $500-$2000 depending on how much it is subsidized by volunteer scientists. We don’t really want Bill Gates etc. to pay for diamond OA. And universities and especially libraries are already overextended. There is no free publishing. The second and, in my opinion most to blame, are the European science grant funders who banded together and came up with Plan S and other schemes to force their scientists to only publish OA. At least in Europe the funding agencies mostly held scientists harmless by paying, and because of the captive audience, publishers went to European countries first for Read and Publish agreements. So European scientists haven’t been hurt too badly. But North America has so far refused to go down the same path, leaving North American scientists without grants (a majority of them) with an ever shrinking pool of subscription-based journals to publish in. And scientists from less rich countries are hurt even worse. Let’s get honest. How long before every university in Africa is covered by a Read and Publish agreement from the for profit companies?

What is interesting about this assessment is that he calls the open access situation a “disaster” on the basis of one very narrow measure: “How does it affect writing scientists?” By “writing scientists,” I mean what are usually called “principle investigators” (PIs), faculty who are busy running a lab and need publications for career advancement.

Two things.

First, most of the paragraph is concerned about how article processing charges affect scientists without grants who need to publish. I emphasize “charges” because, as I have said before, we need to separate open access – a description of who can read scientific articles – from the business models used to support open access. McGill is complaining about the latter, and isn’t addressing the former.

I do agree that many researchers have unrealistic expectations about the costs of publication. I agree that there has not been enough discussion about alternative business models for open access.

Second, journal articles do not just exist merely for the benefit of scientists who need publications to get promotion or tenure. There are not only people who write articles, there are people who read journal articles. You should consider the sizable benefits of more people being able to read scientific papers before judging the success of open access.

Article processing charges do create barriers for researchers with limited resources. But the research of hypothetical African scientists is impeded by not being able to read the scientific literature, not just by being unable to publish in the scientific literature.

If we are concerned about African researcher not being able to pay article processing charges, should we not also be concerned about African researchers being able to buy journal articles or African research libraries being able to buy journal subscriptions?

I see increased ability to read the world’s scholarly literature as a good thing. I don’t see it as an unalloyed good that must be pursued above all else. But it should be in the mix as we’re taking stock of open access.