21 February 2024

I told you transcript changes didn’t affect grade inflation

Way back when, I blogged about a Texas proposal to include average course grades next to a student’s earned grades on the student transcript. The argument was that this could be a way to curb grade inflation. I was skeptical. 

This never came to pass in Texas, but what I didn’t know at the time that this was the practice at Cornell University.

A practice they just stopped.

It turned out that – surprise! – showing average class grades didn’t stop grade inflation. In fact, showing class averages probably increased grade inflation. Because with easy access to average course grades, students preferentially took the classes seen to be “easy A’s”.

I have to admit I didn’t see that possibility, but it tracks.

Related posts

The “Texas transcript” is a good idea, but won’t solve grade inflation

External links

Cornell Discontinues Median Grade Visibility on Transcripts 15 Years After Inception  


19 February 2024

Rats, responsibility, and reputations in research, or: That generative AI figure of rat “dck”

Say what you will about social media, it is a very revealing way to learn what your colleagues think.

Last week, science Twitter could not stop talking about this figure:

Figure generated by AI showing rat with impossibly large genitalia. The figure has labels but none of the letter make actual words.

There were two more multi-part figures that are less obviously comical but equally absurd.

The paper these figures were in has now been retracted, but I found the one above in this tweet by CJ Houldcroft. You can also find them in Elizabeth Bik’s blog.

This is clearly a “cascade of fail” situation with lots of blame to go around. But the discussion made me wonder where people put responsibility for this. I ran a poll on Twitter and a poll on Mastodon asking who came out looking the worst. The combined results from 117 respondents were:

Publisher: 31.6%
Editor: 30.8%
Peer reviewers: 25.6%
Authors: 12.0% 

I can both understand these results to some degree and have these results blow my mind 🤯 a little. 

People know the name of the publisher, and many folks have been criticizing Frontiers as a publisher for a while. Critics will see this as more confirmation that Frontiers is performing poorly. So Frontiers looks bad.

The editor and peer reviewers look bad because, as the saying goes, “You had one job.” They are supposed to be responsible for quality control and they didn’t do that. (Though one reviewer sad he didn’t think the figures were his problem, which will get its own post over on the Better Posters blog later.)

But I am still surprised that the authors are getting off so lightly in this discussion. It almost feels like blaming the fire department instead of the arsonist.

At the surface level, the authors did nothing technically wrong. The journal allows AI figures if they are disclosed, and the authors disclosed it. But the figures are so horribly and obviously wrong that to even submit it feels to me more like misconduct than sloppiness.

And is so often the case, when you pull at one end of a thread, it’s interesting to see what starts to unravel.

Last author Ding-Jun Hao (whose name also appears in papers as Dingjun Hao) has had multiple papers retracted before this one (read PubPeer comments on one retracted paper), which a pseudonymous commenter on Twitter claimed was the work of a papermill. Said commenter further claimed that another paper is from a different papermill.

Lead author Xinyu Guo appears to have been author on another retracted paper.

I’ve been reminded of this quote from a former journal editor:

“Don’t blame the journal for a bad paper. Don’t blame the editor for a bad paper. Don’t blame the reviewers for a bad paper. Blame the authors for having the temerity to put up bad research for publication.” - Fred Schram in 2011, then editor of Journal of Crustacean Biology

Why do people think the authors don’t look so bad in this fiasco?

I wonder if other working scientists relate all to well to the pressure to publish, and think, “Who among us has not been tempted to use shortcuts like generative AI to get more papers out?”

I wonder if people think, “They’re from China, and China has a problem with academic misconduct.” Here’s an article from nine years ago about China trying to control its academic misconduct issues.

I wonder if people just go, “Never heard of them.” Hard to damage your reputation if you don’t have one.

But this strategy may finally be too risky. China has announced new measures to improve academic integrity issues, which could include any retracted paper requiring an explanation. And the penalties listed could be severe. Previous investigations of retractions in China resulted in “salary cuts, withdrawal of bonuses, demotions and timed suspensions from applying for research grants and rewards.”

Related posts

The Crustacean Society 2011: Day 3

References

[Retracted] Guo X, Dong L, Hao D, 2024. Cellular functions of spermatogonial stem cells in relation to JAK/STAT signaling pathway. Frontiers in Cell and Developmental Biology 11:1339390. https://doi.org/10.3389/fcell.2023.1339390

Retraction notice for Guo et al.

External links

Scientific journal publishes AI-generated rat with gigantic penis in worrying incident

Study featuring AI-generated giant rat penis retracted, journal apologizes 

The rat with the big balls and the enormous penis – how Frontiers published a paper with botched AI-generated images

China conducts first nationwide review of retractions and research misconduct (2024)

China pursues fraudsters in science publishing (2015)

29 January 2024

A JIF myth

“Why do zebras have stripes?”

“Why did T. rex have such small arms?”

In evolutionary biology, functional questions like those are notoriously tricky to answer, because people tend to mix up two separate questions: the origin of the feature, and the current use of the feature.

People often have difficulty grasping that those two things are different. This leads to questions like, “What good is half a wing?” . The implication is that that because wings are used for flying now, they must always have been used for flying, so how could they evolve?

The answer is that the bits that make up a wing can be used for lots of things, and that you can have a functional shift. Something that’s great for insulation or display like feathers proves incidentally useful for gliding. The incidental use eventually becomes the primary use.

Clarivate Analytics Journal Impact Factor
I mention this because I stumbled across a myth about the Journal Impact Factor™️. You will find statements here and there Eugene Garfield created the Impact Factor to help libraries decide what journals to buy. This was back in the day when journal subscriptions were sold individually rather than in bundled “big deals.”

But Garfield’s own account of the history of the Impact Factor shows this is not true (link to edited version; full version can be found in external links).

Garfield was involved in creating the Genetics Citation Index in the 1950s, and needed to decided what journal to include in the index. They first tried just counting citations to journals, which favoured journals that just published a lot of papers. They realized that “total citations” missed journals that published fewer papers that were highly cited.

Impact Factor helped solve this problem. It wasn’t about libraries at all. So where did the “Impact Factor was created for libraries” belief come from?

In the 1960s, the Genetics Citation Index broadened out to become the Science Citation Index. Garfield did lots of research on citation patterns, and published a 1972 paper in Science about citation analysis. Garfield talks about all the findings using this measure, Impact Factor, that he created more than a decade earlier.

But at the end of the paper, there’s a section that begins, “Some applications,” and the first sentence is:

The results of this type of citation analysis would appear to be of great potential value in the management of library journal collections.

And that’s the origin of the myth that Impact Factor was created for libraries.

That 1972 Science paper went on to become the one that many people referred to in discussions of Impact Factor. And it’s easy to see how Garfield’s suggestion that libraries could use Impact Factor to make purchasing decisions could morph into, “Garfield created Impact Factor for libraries.” Because people don’t always read the original papers of the things they cite in details.

The Impact Factor not only shows how easily origin gets muddled, it also shows the concept of functional shift. Because libraries did use it for purchasing decisions, and then administrators started using it to make hiring, tenure, and promotion decisions. DORA is trying to provide a new selection pressure that could cause another functional shift, away from evaluating researchers.

External links

Essays about Impact Factor by Eugene Garfield

Garfield E. 1972. Citation analysis as a tool in journal evaluation. Science 178: 471-479. https://doi.org/10.1126/science.178.4060.471 

17 January 2024

Anonymous Predatory Reports site is predatory

Some time ago, I wrote about the anonymous website Predatory Reports. Some things about it struck me as sketchy

It looks like my instincts were on the money. 💵 

The commercial journal vetting service Cabells is reporting major misdeeds from Predatory Reports. Specifically that:

  1. The site has a bunch of plagiarized material.
  2. The site owner tried to squeeze money out of Cabells.
I doubt this story is done. This is, I think, only the second act. Since the story isn’t done. I don’t know if it’s appropriate to have a moral quite yet, but it might be one of the academic Moscow rules: always trust your gut.

Hat tip to Anna Abalkina.

External links

Unmasking a predator: PredatoryReports.org

Related posts

A star chamber for predatory journals

07 January 2024

Kung Fu Panda is going to ruin my life

I’m at the movies last night, watching the previews, and up comes a blurb about Kung Fu Panda 4.

Now I have to tell you, I love the Kung Fu Panda movies. So I’m excited!

And the presenter says, “We know there’s going to be a new character called ‘Zhen.’”  (The ‘h’ is silent, at least as pronounced by the presenter.)

Zhen from Kung Fu Panda 4

And as you can see, this character is a fox.

Zhen the fox.

Zhen. Fox.

I must now accept that this is what people are going to think of when they hear my name for years.

I suppose it’ll make for a change from motorcycle repair jokes, though.

26 December 2023

Cell Bio 2023

How it started:


How it’s going:


Second photo from https://x.com/lenakumba/status/1731423282228867322

26 December 2023: Originally posted on 3 December 2034 on the Better Posters blog by accident.

23 September 2023

A biologist in a physics mag

Physics Today cover for October 2023The change in my job is slowly starting to have an effect. I’m quoted, as DORA Program Director, in a new article in Physics Today.

As a biologist, I have to say that this is not ever something I aspired or expected in my career. So, that’s interesting.

References

Feder T. 2023. Global movement to reform researcher assessment gains traction. Physics Today 76(10): 22–24. https://doi.org/10.1063/PT.3.5323