11 May 2019

A pre-print experiment, part 3: Someone did notice

In 2016, I wrote a grumpy blog post about my worries that posting preprints is probably strongly subject to the Matthew effect. It was a reaction to Twitter anecdotes about researchers (usually famous) posting preprints and immediately getting lots of attention and feedback on their work. I wanted to see if someone less famous (i.e., me) could get attention for a preprint without personally spruiking it extensively on social media.

I felt my preprint was ignored (until I wrote aforementioned grumpy blog post). But here we are a few years later, and I’m re-evaluating that conclusion.

A new article about bioRxiv is out (Abdill and Blekman 2019), and it includes Rxivist, a website that tracks data about manuscripts in bioRvix. Having posted a paper in BioRvix, that means that my paper is tracked in Rxivist.

It’s always interesting to be a data point in someone else’s paper.

The search function is a little wonky,but I did find my paper, and was surprised (click to enlarge).


Rxivist showed that there has been a small but consistent number of downloads (Downloaded 421 times). Not only that, but the paper is faring pretty well compared to others on the site.
  • Download rankings, all-time:
    • Site-wide: 17,413 out of 49,290
    • In ecology: 542 out of 2,046
  • Since beginning of last month:
    • Site-wide: 19,899 out of 49,290
My little sand crab natural history paper is in the top half of papers in bioRxiv?

I did not expect that. Not at all.

I know there is an initial spike because I wrote my grumpy blog post and did an interview about preprints that got some attention, but even so. I know there aren’t hundreds of people doing research on sand crabs around the word, so hundreds of downloads is a much wider reach than I expected.

And some of the biggest months (October 2018) are after the final, official paper was published in Journal of Coastal Research. The final paper is open access on the journal website, too, so it’s not as though people are downloading the preprint because they are circimventing paywalls. (Though in researching this blog post, I learned a secondary site, BioOne, is not treating the paper as open access. Sigh.) (Update, 14 May 2019: BioOne fixed the open access problem!)

I am feeling much better about those numbers now than in the first few months after I posted the paper. I never would have anticipated that long tail of downloads years after the final paper is out.

And Rxivist certainly does a better job of providing metrics than the journal article does:


There’s an Altmetric score but nothing else. It’s nice that the Altmetric score for the preprint and published paper are directly comparable (and I’m happy to see the score of 24 for the paper is a little higher than the preprint at 13!), but I miss the data that Rxivist provides.

Other journals provide top 10 lists (and I’ve been happy to be on those a couple of times), but they tend to be very minimal. You often don’t know the underlying formula for how they generate those lists. The Journal of Coastal Research has a top 50 articles page that shows raw download numbers for those articles, and if you are not in that list, you have no idea how your article is doing.

While I still never got any feedback on my article before publication, I don’t feel like posting that preprint was a waste of time like I once did.

References

Abdill RJ, Blekhman R. 2019. Tracking the popularity and outcomes of all bioRxiv preprints. eLife 8: e45133. https://doi.org/10.7554/eLife.45133

Faulkes Z. 2016. The long-term sand crab study: phenology, geographic size variation, and a rare new colour morph in Lepidopa benedicti (Decapoda: Albuneidae). BioRXiv https://doi.org/10.1101/041376

Faulkes Z. 2017. The phenology of sand crabs, Lepidopa benedicti (Decapoda: Albuneidae). Journal of Coastal Research 33(5): 1095-1101. https://doi.org/10.2112/JCOASTRES-D-16-00125.1 (BioOne site is paywalled; open access at https://www.jcronline.org/doi/full/10.2112/JCOASTRES-D-16-00125.1)

Related posts

A pre-print experiment: will anyone notice?
A pre-print experiment, continued

Fiddly bits and increments

External links

Sand crab paper on Rxivist

10 May 2019

Reliable shortcuts

There’s an old saying that if you build a better mousetrap, the world will beat a path to your door.

I think that’s true of shortcuts, not mousetraps.


Everyone wants reliable shortcuts. They don’t want to have to assess every available option every single time.

Best seller lists, Consumer Reports, “Two thumbs up!” from Siskel and Ebert, “Certified fresh” on Rotten Tomatoes, “People also shopped for” on Amazon, “Five stars” on Yelp!, and awards shows are all efforts to create shortcuts.

An amazing number of arguments in academia are about shortcuts. Almost every debate about tenure and promotion and assessments of academics I have seen or read is about shortcuts. Arguments about the GRE are about shortcuts.

I got thinking about shortcuts because of this article on what journals go into PubMed.

For some members of the scientific community, the presence of predatory journals, publications that tend to churn out low-quality content and engage in unethical publishing practices—has been a pressing concern.

Rebecca Burdine is among the concerned, because she advised people to use PubMed as a shortcut.


I could tell parents “researching” their rare disease of interest that if it wasn’t on PubMed, then it shouldn’t be given lots of weight as a source.

Stephen Floor thinks the problem is even wider:

This has also contributed to the undermining of “peer reviewed” as a measure of validity.

But again, “peer reviewed” is a shortcut. Anyone who’s been in scientific publishing for a while knows that assessing scientific evidence is messy and complicated. Every working scientist has their own “That should never have gotten past peer review!” story.

We will never, ever get rid of shortcuts. People crave certainty and simple decision making rules. But we should talk about using shortcuts in science in realistic ways.

It is not reasonable to expect any shortcut to be perfectly reliable all the time. Don’t ask, “Which shortcut is better?” but “How can I use a few different shortcuts?”

Unfortunately, scientists who understand the nuances of a situation often do a shoddy job of conveying that nuance. Or maybe they just get tired of being pressed for shortcuts. So we have kind of brought this on ourselves.

External links

Academics Raise Concerns About Predatory Journals on PubMed


25 April 2019

Rejected for literary allusions

Ton van Raan had a paper rejected for referring to “sleeping beauty.” This is a term that van Raan and others have used to describe papers that aren’t cited for a long time and then start accumulated citations years after publication.

Rejection always makes academics grumpy, and van Raan is grumpy about this, unleashing the “political correctness gone mad” trope in his defense.

I went through something similar myself. I know first hand how easy it is to get defensive about deploying a common metaphor. But after you calm down and think about it, there is often a fair point to the criticism.

I think the editor had a good criticism but a bad implementation. Sleeping Beauty is problematic. Like many fairy tales, modern North Americans tend to know only very sanitized versions of the story. But even in the “clean” version, the part everyone remembers is pretty creepy.

Sleeping Beauty saying to Prince, "Whoa, what part of me sleeping here alone implied consent?"

I think the editor was right to ask for the author to not use Sleeping Beauty as a metaphor for scientific papers. Maybe a passing comment that this is how they have been referred to in the literature would be fine.

The editor may have failed in two ways.

The first was in rejecting the paper for a bad metaphor alone. If the scientific content was correct, it would seem that “Revise and resubmit” might have been a more appropriate response. It seems churlish to chuck the paper entirely for a poor metaphor, particularly one that is, for better or worse, already used by others in the literature.

The second was not offering a positive alternative to the Sleeping Beauty reference. Maybe these papers could be “buried treasures” or something that might be less problematic. There are many ways to express ideas. And neither an editor nor an author should dig in their heels over any particular way of expressing an idea.

Both of these problems assume the account offered by van Raan is accurate. Maybe there were other reasons for rejecting the paper besides the fairy tale reference. We don’t know.

Criticism can be valuable. Criticism plus suggestions for corrections are even better.

Hat tip to Marc Abrahams.

Related posts

Wake up calls for scientific papers

External links

Dutch professor astonished: comparison with Sleeping Beauty leads to refusal of publication

Scientists’ unguarded moments

Earlier on Twitter, I shared an Instagram picture of Dr. Katie Bouman at work, imaging a black hole.


This has been a widely shared picture, and I was a little surprised when I saw a friend on Facebook question it. She said when there was scientific discoveries that showed images of men, the men looked more composed and professional. This made the imaging of a black hole look like an almost accidental “Did I do that?” moment.

Certainly Bouman’s picture was not the only one available. Here’s a picture of another one of the black hold team, Kazunori Akiyama.


Both Akiyama and Bouman are at computers, looking at the historic images they made. But Akiyama’s is almost certainly a staged, posed picture.

I asked what she thought of this picture of the New Horizons team, looking at images of Pluto close up for the first time. The man in the middle is project leader Alan Stern.


She replied that she thought it did a disservice to the science and to Stern if it had been widely reported. It had been seen quite widely, though probably not as much as the Bouman picture.

This interested me, because it spoke to the risks and rewards of scientists showing their unguarded moments.

On the one hand, these spontaneous moments capture something that is, I think, deeply human. Excitement. Joy. Achievement. Surprise.

But I also get that these are moments where people look undignified or vulnerable. It’s easy to mock people for looking goofy. Especially for big projects that have a lot of taxpayer money behind them, it might not be a good look. It can look like people just screwing around.

Personally, I think that sharing those spontaneous moments are worth it. My wife has been watching a lot of Brené Brown talks recently, and she talks a lot about the concept of vulnerability. And how vulnerability is one of the best predictors of courage.

I think scientists could use a lot more encouragement. And if that means looking in a way that surprises people, that’s all right by me.

Since I’m talking black holes, there was some discussion over who did what. Bouman got a lot of media attention, Sara Issaon pointed out that the image was not Bouman’s algorithms alone. Weirdly, this snowballed into some trying to undermine her contribution entirely. Andrew Chael ably tackled the trolls.

References

First M87 Event Horizon Telescope Results. IV. Imaging the Central Supermassive Black Hole

External links

Meet one of the first scientists to see the historic black hole image


12 April 2019

“Open access” is not synonymous with “author pays”

Wingfield and Millar have a well-meaning but misleading article, “The open access research model is hurting academics in poorer countries.” They say:

The open access model merely changes who pays. So rather than individuals or institutions paying to have access to publications, increasingly, academics are expected to pay for publishing their research in these “open access” journals. ... The bottom line is that payment has been transferred from institutions and individuals paying to have access to researchers having to pay to have their work published.

The first sentence is correct. The second is even correct. It is true that there are now more journals that require article processing charges than their used to be. Importantly, though, the phenomenon of authors paying is not new. “Pages charges” existed long before open access.

But they lose all nuance in the third sentence and commit a category error. They are confusing “freedom to read” with “business model.” These two things are not the same.

There are many counter examples to their central premise. SciELO journals are open access, but have no article processing fees. I could go on.

I am not saying that there is not a concern about the effects of article processing charges. It isn’t even restricted to scientists in “poorer countries.” Michael Hendricks, a biologist at one of Canada’s major research universities (hardly a “poorer country” by any measure, and not even a “poorer institution” by any measure) is concerned about the cost of article processing charges. He wrote:

US$2500 is 1% of an R01 modular budget. It is 2.5% of the average CIHR Project grant. It’s 10% of the average NSERC grant.

Add to that the vastly differing support across universities for article processing charges (ours is $0). There is no way around that fact that shifting publication costs from libraries to PIs imposes a massively different burden according to PI, field of science, nation, and institution.

The solution is that universities should pay article processing charges by cancelling subscriptions (with huge $ savings). But they generally aren’t. The only way I see to force the issue is for funders to make article processing charges ineligible, which will be seen as an attack on open access.

It’s real problem: library subscription costs are staying the same or going up. At the same time, more and more grant money is being spent on article processing charges. The public paying even more for science dissemination than they were is not what we want. Funders and/or universities have to stop this.

But looking back up to the counter-example, SciELO, shows something important. It shows that you can create open access journals with alternative business models that are not “author pays.” It’s unusual, maybe even difficult, but it’s not impossible.

That’s a line we should be pursuing. Not dumping on open access because people can’t distinguish between “common” and “necessary.”

External links

The open access research model is hurting academics in poorer countries

03 April 2019

Recommendation algorithms are the biggest problem in science communication today


Having been interested in science communication (and being a low-level practitioner) for a while, I recognized that there are a lot of old problems that recycle themselves. Like, “Should we have scientists or journalists communicating science to the public?” (Why not both?) “Is there value in debunking bad science, or does it make people dig in and turn off?” “Why can’t people name a living scientist?” “You have to tell a story...”

Having followed social controversies about science (particularly the efforts of creationists to discredit evolution) for a long time, I was almost getting bored seeing the same issues go ‘round and ‘round. But now I think we are truly facing something new.

Recommendation algorithms on social media.

You know these. These are the lines of computer code that tells you what books you might like on Amazon based on what you’ve bought in the past. It’s the way Facebook and Instagram deliver ads that you kind of like seeing in your feed. And it’s how Netflix and YouTube shows you want you might want to watch next.

I think recommendation algorithms may be the number one problem facing science communication today.

And of these, YouTube seems to be particularly bad. How bad is YouTube? Pretty bad, in that it is very effective at convincing people of false stuff.

Interviews with 30 attendees revealed a pattern in the stories people told about how they came to be convinced that the Earth was not a large round rock spinning through space but a large flat disc doing much the same thing.

Of the 30, all but one said they had not considered the Earth to be flat two years ago but changed their minds after watching videos promoting conspiracy theories on YouTube.

You watch one thing, and YouTube recommends something even a little crazier and more extreme. Because YouTube wants you to spend more time on YouTube. People are calling this “rabbit hole effect.”

It’s not just happening over scientific facts, either. Many have noted that YouTube is having a similar effect in politics, with many branding it a “radicalization machine.”

Reporter Brandy Zadrozny summarizes YouTube’s defense thus:

YouTube’s CPO says the rabbit hole effect argument isn’t really fair because while sure, they do recommend more extreme content, users could choose some of the less extreme offerings in the recommendation bar.

So we need humans need to fix the problem, right? Human content moderation is the answer! Well, maybe not, because repeated exposure to misinformation makes you question your world view.

Conspiracy theories were often well received on the production floor, six moderators told me. After the Parkland shooting last year, moderators were initially horrified by the attacks. But as more conspiracy content was posted to Facebook and Instagram, some of Chloe’s colleagues began expressing doubts.

“People really started to believe these posts they were supposed to be moderating,” she says. “They were saying, ‘Oh gosh, they weren’t really there. Look at this CNN video of David Hogg — he’s too old to be in school.’ People started Googling things instead of doing their jobs and looking into conspiracy theories about them. We were like, ‘Guys, no, this is the crazy stuff we’re supposed to be moderating. What are you doing?’”

Mike Caulfield noted:

I will say this until I am blue in the face – repeated exposure to disinformation doesn’t just confirm your priors. It warps your world and gets you to adopt beliefs that initially seemed ridiculous to you.

Propaganda works. Massive disinformation campaigns work. Of course, people with a point of view and resources have known this for a long time. Dana Nucitelli noted:

(T)he American Petroleum Institute alone spent $663 million on PR and advertising over the past decade - almost 7 times more than all renewable energy trade groups combined.

Meanwhile, scientists are still hoping that just presenting facts will win the say. I mean, the comments coming from the first day of a National Academy of Sciences colloquia feel much like the same old stuff, even though the title seems to hint at the scope of the problem (“Misinformation About Science in the Public Sphere”). It feels like the old arguments about the best way that individual scientists can personally present facts, ignoring the massive machinery that most people are connected to very deeply: the social, video, and commercial websites that are using recommendation algorithms to maximize time people spent on site.

The good news is that as I was writing this, the second day seems to be much more on target. And one speaker is saying that the biggest source of news is still television. Sure, but what about all the other information people get that is not news?

The algorithm problem requires a deep reorienting of thinking. I don’t quite know what that is yet. It is true that this is a technological problem, and technological fixes may be relevant. But I think Danah Boyd is right that we can’t just change the algorithms, although I think changing algorithms is necessary. We have to change people, too, beyond a “If they only knew the facts” kind of way. Because many do know the facts, and it don’t settle matters. But changing culture is hard.

I think combating the algorithm problem might require strong political action to regulate YouTube in the way television networks in the US were (and, in other countries, still are) regulated. But the big social technology companies are spending millions in lobbying efforts in the US.

Update, 17 May 2019: A hopeful tweet on this matter.

Promising data point for YouTube’s anti-conspiracy push (on 1 topic, at least): the people in my flat-earth FB group are very mad that the site has stopped shoveling lunatic videos into their feeds.

External links

The trauma floor
Study blames YouTube for rise in number of Flat Earthers
Google and Facebook can’t just make fake news disappear

YouTube’s algorithms can drag you down a rabbit hole of conspiracies, researcher finds
Google, Facebook set 2018 lobbying records as tech scrutiny intensifies
Americans are smart about science

19 March 2019

The legality of legacy admission

In the light of the “Operation Varsity Blues” college scandal last week, a lot of people were complaining about university admissions generally. I learned that a lot of people:

  1. Think university admissions are hopelessly corrupt across the board, and that these cases were not “a few bad apples.”
  2. Are super grumpy about “legacy admission.” 

I knew about court cases  about affirmative action (including the current one at Harvard), but I got curious as to whether legacy admissions had ever faced a legal challenge, and if so, what was the basis for keeping it.

I found one case that concerned legacy admissions: Rosenstock v. The Board of Governors of the University of North Carolina. This is the relevant bit about legacy admissions:

Plaintiff also attacks the policy of the University whereby children of out-of-state alumni are exempted from the stiffer academic requirements necessary for out-of-state admission. Again, since no suspect criteria or fundamental interests are involved, the State need only show a rational basis for the distinction. In unrebutted affidavits, defendants showed that the alumni provide monetary support for the University and that out-of-state alumni contribute close to one-half of the total given. To grant children of this latter group a preference then is a reasonable basis and is not constitutionally defective. Plaintiff's attack on this policy is, therefore, rejected.

The questions raised here are, in large part, attacks on administrative decision-making, an area where the federal courts have not and should not heavily tread. Plaintiff has not shown a constitutional reason for abandoning this judicial policy.

The court is saying legacy admissions are okay because the university can make money. And it’s not up to courts to change administrative decisions.

Regardless, I kind of suspect that legacy admissions are going to come under increasing pressure because they are, as the pundits say, “a bad look” for universities.

External links


Six of the top 10 universities in the world no longer consider legacy when evaluating applicants—here’s why

What we know so far in the college admissions cheating scandal

18 March 2019

The Zen of Presentations, Part 72: Hasan Minhaj is one of the best presenters today

At any given moment in time, there are people who are well known for giving good presentations.

In the early part of the twenty-first century, many people pointed to Steve Jobs as an example of what a great presenter could do. In her book Resonate Nancy Duarte says, “Jobs had the uncanny ability to make audience engagement appear simple and natural.” She points to the iPhone launch in 2007 as one of the best product launches of all time.

I often pointed to Hans Rosling, who leapt into people’s awareness with some of the first TED talks in 2006. Indeed, Rosling practically provided the templatefor what a TED talk was. Others followed in his footsteps for years to come.

But we lost Jobs in 2011, and Rosling in 2017.

But now I would like to nominate the person who is, I think, one of the best presenters of this time.


Hasan Minhaj.

You might object that Minhaj is a stand-up comedian, and stand up isn’t really a presentation in the usual sense. That’s certainly what I might have thought when I had only seen him on The Daily Show. Funny, yes. But a great presenter?

But then I saw his special Homecoming King. It’s stand up, but like many one person shows, there’s a strong narrative running though it. It mostly revolves around a prom date gone wrong.


But it’s not just Minhaj on a stage. He has a screen that shows a lot of images that are relevant to what he is describing. In other words, his Peabody Award winning special is a PowerPoint presentation. A high end and heavily disguised PowerPoint presentation, but it’s not such a different beast than many.

His Netflix series Patriot Act is less personal but more topical, and Minhaj pushes his presentation skills even further. In each episode, Minhaj does a deep explanation of one or two subjects. In science communication terms, Minhaj is making “explainers.”



And these are data driven episodes on somewhat esoteric subjects. You don’t see a lot of coverage of the Indian general elections in the news on North America.



Chinese censors, street wear hype, drug pricing, and affirmative action all come under the microscope. (In light of the university admissions scandal that broke last week, the first episode about university admissions is worth a watch, too, as Minaj lays out the the background for the lawsuit against Harvard about admissions that is being backed by white guys trying to destroy affirmative action.)

Patriot Act the only show I can think of that wouldn’t surprise me if it did an entire episode about Plan S and academic publishing.

Why I think Minhaj’s presentation is the best around right now?

Obviously, Minhaj is legit funny. But he isn’t afraid to tell niche joke. In one episode, he says something like, “I tell jokes for four people at a time.”

Minhaj’s show is committed to evidence and data. Minhaj says he has a team of researchers that help him look smart, but most shows wouldn’t bother. Most comedy shows would just be content to have their comedian mouth off whatever thoughts they have, maybe with some light fact checking. But Minhaj is not just expressing opinions. He’s building arguments.

Minhaj is concise, and has the ability to sum up complicated backstory in a few short, well-chosen sentence. Almost accidentally, this makes him fast. I sometimes think an episode of his show would almost be one of the best “Intro to political science”lectures on any campus, but then I realize that it would be too quick for students to take notes. But you’re not taking notes, so it doesn’t matter. You can just enjoy the delivery and flow.

And Patriot Act is filmed in front of an audience. While his monologues are obviously incredibly tightly scripted, Minhaj still pays attention to his audience. He goes off script for a few seconds to responds to them and interact with them.

While I said Minhaj’s lectures wouldn’t be too effective for students trying to take notes, I will be taking notes: not on the content, but to figure out what makes his presentations so good.