16 August 2018

How to present statistics: a gap in journals’ instructions to authors

I recently reviewed a couple of papers, and was reminded by how bad the presentation of statistics in many papers is. This is true even from veterans with lengthy publication records who you might think would know better. Here are a few tips on presenting statistics that I’ve gleaned over the years.

Averages


One paper wrote “(7.8 ± 0.8).” I guess this was supposed to be a mean and standard deviation. But I had to guess, because the paper didn’t say. But people often report other measures of dispersion around an average (standard errors, coefficient of variations) the exact same way.

Curran-Everett and Benos (2004) write, “The ± symbol is superfluous: a standard deviation is a single positive number.” When I tweeted this yesterday, a few wrote that this was silly, because Curran-Everett and Benos’s recommended format was a few characters longer and people worried about it being repetitive and hard to read. This reminds me of fiction writers who try to avoid repeating “He said,” usually with unreadable results. My concern is not about the ± symbol as the numbers having no description at all.

Regardless, my usual strategy is similar to Curran-Everett and Benos. I usually write something like, “(mean = 34.5, SD = 33.0, n = 49).” Yes, it’s longer, but it’s explicit.

That strategy isn’t the only way, of course. I have no problem with a line saying, “averages are reported as (mean ± S.D.) throughout.” That’s explicit, too.

Common statistical tests


Another manuscript repeatedly just said, “p < 0.05.” It didn’t tell me what test was used, nor any other information that could be used to check that it is correct.

For reporting statistical tests, I write something like, “(Kruskal-Wallis = 70.76, df = 2, p < 0.01).” That makes it explicit:

  • What test I am using.
  • The exact test statistic (i.e., the result of the statistical calculations).
  • The degrees of freedom, or sample size, or any other values that is relevant to checking and interpreting the test’s calculated value.
  • The exact p value, and never “greater than” or “lesser than” 0.05. Again, anyone who wants to confirm that the calculations have been done correctly needs an exact p value.  People’s interpretations of the data change depending on the reported p value. People don’t interpret a p value of 0.06 and 0.87 the same, even though both are “greater than 0.05.” Yes, I know that people probably should not put much stake in that exact value, and that p values are less reproducible than people expect, but there it is.

My understanding is that these values particularly matter for people doing meta-analyses.

Journals aren’t helping (much)


I wondered why I keep seeing stats presented in ways that are either uninterpretable or unverifiable. I checked the author’s instructions of PeerJ, PLOS ONE, The Journal of Neuroscience, and The Journal of Experimental Biology. As far as I could find, only The Journal of Neuroscience provided guidance on what their standards for reporting statistics is. The Journal of Experimental Biology’s checklist says, “For small sample sizes (n<5), descriptive statistics are not appropriate, and instead individual data points should be plotted.”

(Additional: PeerJ does have some guidelines. They are under “Policies and procedures” rather than “Instructions for authors,” so I missed them in my quick search.)

This stands in stark contrast to the notoriously detailed instructions practically every journal has for reference formatting. This is even true for PeerJ, which has a famously relaxed attitude towards reference formatting.

In the long haul, the proper reporting of statistical tests is probably more important to the long term value of a paper in the scientific record than the exact reference format.

Judging from how often I see minimal to inadequate presentation of statistics in manuscripts that I’m asked to review, authors need help. Sure, most authors should “know better,” but journals should provide reminders even for authors who should know this stuff.

How about it, editors?

Additional: One editor took me up on this challenge. Alejandro Montenegro gets it. Hooray!

More additional: And Joerg Heber gets it, too. Double hooray!

References

Curran-Everett D, Benos DJ. 2004. Guidelines for reporting statistics in journals published by the American Physiological Society. American Journal of Physiology - Gastrointestinal and Liver Physiology 287(2): G307-G309. http://ajpgi.physiology.org/content/287/2/G307.short

06 August 2018

Maybe we can't fix “fake news” with facts

There’s been a few recent moves in the war on “fake news.” For instance, several platforms stopped hosting a certain conspiracy-laden podcast today. (You can still get all that conspiratorial stuff. The original website is untouched.) But the discussion about “fake news” seems to be focusing on one thing: its content.

Lately, I’ve been thinking about this diagram I made about communication, based on Daniel Kahneman’s work. Kahneman argues you need three things for successful communication:

  1. Evidence
  2. Likeability
  3. Trust

I feel like most of the talking about “fake news” is very focused on “evidence.” This article, for instance, describes some very interesting research about how people view news articles. It’s concerned with how people are very prone to value opposing sources, but are very poor at evaluating the credibility of those sources.

All good as far as it goes. But, as I mentioned before, it feels a lot like science communicators who, for years and years, tried to beat creationists, flat Earthers, anti-vaccine folks, and climate change deniers by bringing forward evidence. They were using the deficit model: “People must think this because they don't know the facts. We must get the facts to them.”

That didn’t work.

I’m kind of seeing the same trend in fighting fake news. “Remove the falsehoods, and all will fix itself.”

But where I see the truly big gap between where we were and where we are isn’t about facts. It’s about trust.

When you bring evidence to a fight that isn’t about facts, you will lose. Every time. Facts mean nothing without trust. “Check the facts” means nothing when you are convinced everyone is lying to you. This is why conspiratorial thinking is so powerful and dangerous: it destroys trust.

You see the results in how someone who buys into one conspiracy theory often buys into several other conspiracy theories. If you believe Obama wasn’t born in the US because conspiracy (say), it’s not that big a leap to the moon landings were fake and the Earth is a flat square.

I have some hypotheses about how America in particular got to this point. I suspect the erosion of trust was slow, gradual, and – importantly – started long before social media. Maybe more like, I don’t know, let’s say 1996.

I don’t know how to reverse a long-term trend of distrust and paranoia. I’m not saying, “We need to understand and sympathize with fascists,” either. But you can’t cure a disease when you misdiagnose it. I just don’t see focusing on the factual content of social media getting very far.

Related posts

The Zen of Presentations, Part 59: The Venn of Presentations
Post fact politics catches up to science communication

External links

Fake news exploits our obliviousness to proper sourcing
Looking for life on a flat Earth
How can I convince someone the Earth is round?
Why do people believe conspiracy theories?

25 July 2018

Crayfish clothing contest conqueror!


You are looking at the winner of the International Association of Astacology T-shirt contest! By me!

  • First place: “Astacus fluviatilis” by Zen Faulkes
  • Second place: “Euastacus,” front and back design by Premek Hamr
  • Third place: “Astacolic” by Alexa Ballinger (which you can see here)

I haven’t yet see the runner-up designs, which were shown at the last IAA meeting in Pitssburgh, but I look forward to seeing them! This started with the quote. I found it on page vi of Thomas Henry Huxley’s monograph on the crayfish (also Google Books edition). It took a little digging to find the author’s complete name and year of the quote. (Yes, my academic training is showing: obsession with complete and correct citations.)

While looking up the person who wrote the quote, I discovered that Rösel von Rosenhof was an amazing illustrator of the natural world. And he painted crayfish! So I was able to combine this wonderful quote about crayfish with this brilliant plate by the same person.

I cleaned up an image of one of Rösel von Rosenhof’s paintings, cleaning up page blemishes left over from the scan and making it a little brighter.

I kept some of the writing on the painting but repositioned it. The quote that started me off was not on the page, so I had to add it. I had just the thing: the Adorn font family evoked the style the old plate well. But the wonderful thing about a well made font family is that you can use a lot of different variations of text, and it still feels coherent.


Adorn has a lot of built in letter variants, and it was fun to play around with different swashes in CorelDraw! I am pleased people like this, but I’m sure that it won the contest is more a tribute to the artistry of Rösel von Rosenhof than my own graphic design skills. But this was not the only T-shirt design I submitted. Oh no. I was having fun with the shirt designs. This was actually the first concept I worked on:


The outline is a signal crayfish claw, if I remember right.The words inside the claw outline are the names of every genus of freshwater crayfish (according to Crandall and de Grave 2017). Originally, I played with the idea of using the name of every species of crayfish, but with over 600 and rising, there were too many and it was too likely to go out of date soon.

I like this design, but I was never able to get it to look exactly like it was in my head. I wanted the shape of the claw to be defined by the words alone. I like the big, bold shape of the claw and that it includes all the crayfish diversity within it, though.

I worried that the genus names might be too small and fussy for a T-shirt, but I liked that claw shape, so I made this variant:


It’s bold, though I worry that it’s a little simple.

To be honest, this was my favourite design:


I traced an image of a crayfish using in CorelDraw. I love Japanese and Chinese calligraphy, and modified the lines making up the trace to taper, giving it a sort of brush-like appearance. The font is Cherry Blossoms, which I wrapped around on an oval. This font, like Adorn, has a lot of options, and I had way too much fun trying out different swashes. (Discovering alternate glyphs and swashes has been a revelation.)

Initially, I only had “International Association of Astacology.” But the words traced out the oval so clearly on the top that the bottom looked broken and incomplete. I needed something to complete the shape, so I added in “the science of crayfish.” I loved this, because I feel like one of the big problems with being a member of the International Association of Astacology is explaining what “astacology” is!

I made variations of the three no-winning designs, too, changing the fonts and colours in different ways.The first version of the brushwork crayfish above had the colours flipped, with the crayfish in red, and the text above in black. But since ink was the inspiration, making the crayfish black made more sense.

Even though my favourite design didn’t win, I am completely thrilled to have won the T-shirt contest. I am mentioning this award this in my annual review folder!

And maybe a few more people will discover and appreciate the fine artwork of Rösel von Rosenhof.

References

Crandall KA, De Grave S. 2017. An updated classification of  the freshwater crayfishes (Decapoda: Astacidea) of  the world, with a complete species list. Journal of Crustacean Biology 37(5): 615-653. https://doi.org/10.1093/jcbiol/rux070

External links

August Johann Rösel von Rosenhof
How to swash: using a font’s alternate glyphs, text styles, and numbers
Critique: The Capricorn Experiment, plus: Font families

17 July 2018

Tuesday Crustie: Crab emojis rated

It is, according to my social media, World Emoji Day. No sure why that needs a day, but I’m not here to judge. Well, not, today I am, because I am here to judge crab emojis!

Apple


I think this crab is cold. That would explain why it seems to be wearing mittens over its claws. 6 out of 10.

EmojiOne


Is this crab doing a shadow puppet play about warring sibling birds in the nest? The claws look like little bird beaks.The eyes seem way to big for its stalks. This is some sort of angsty, emo teenage crab with an art project. It might be nice when it grows out of its akward phase. 3 out of 10.

Facebook


Sure, this social media giant is contributing the decline and probable fall of democracies in several nations, but damn! They got someone who was paying attention in invertebrate biology and respects the crustacean to draw their emoji. The claws look like they could do serious damage. The lines on the carapace show more attention to detail than anyone else. I feel like I could almost key it out to the genus. 10 out of 10.

Google


The Google crab has seen your browser history and is shocked, shocked it says, by the websites you visit. It wants to push you away, which is why its claws are in some sort of weird backwards pose. 4 out of 10.

LG


An abyssal crab, judging from the teensy-weensy eyes. Kind of funky bulbous claws, but bonus points for its wickedly curved final pair of thoracopods. They look like kama. 7 out of 10.

Microsoft


I think this crab was produced by a game of telephone between artists. Someone once saw a crab, drew it, then showed their drawing to someone who copied the drawing, who showed it to another person who copied the drawing, and this went on maybe about ten to twenty times. As Magritte might say, “Ce n’est pas un crabe.” (This is not a crab.) 3 out of 10.

Samsung


Argh! Dude! What happened to your legs? And your claws can’t open! I’m so sorry. Is there a foundation we can donate for to support research into your condition? 1 out of 10.

Emojidex

Flashback to 1984 and the release of Romancing the Stone:
Gloria: [observing men in a bar]

Wimp. Wimp. Loser. Loser. Major loser. Too angry. Too vague. Too desperate. God, too happy.


This is the “God, too happy” crab. Seriously freaking me out how happy this guy is. I think this crab has had chemical stimulation. -2 out of 10.

Twitter


Another poor unfortunate victim of some sort of leg disfigurement. Those claws cannot work, the leg tips look unsuitable for grasping any benthic substratum. And yet I can feel somethings besides pity for this crab. Something about the dots on the carapace say to me, “This is a crab that has not stopped living. This is a crab happy to get a tramp stamp and put it out there.” 6 out of 10.

WhatsApp


Ooh, it’s a little baby crab! It’s so tiny! It looks like it lives in the water column, floating and hoping no fish eats it. It will be sad when it grows up and will turn into into the EmojiOne artsy emo teen crab. 8 out of 10.

What have we learned from all this? That all crabs in the Internet are red. Or orange-y red. Except for Samsung crab, whose brown colour is obviously some sort of symptom of whatever disease it has.

Inspiration from ant emoji ratings. Hat tip to Alex Wild and Melanie Ehrenkranz.

Come back for World Emoji Day 2019, when we’ll rate shrimp emoji!

External links

an entomologist rates ant emojis

10 July 2018

Ego

Hypothesis:

Complaints about peer review are often made by people who believe that their work is so infallible and perfect that it cannot be made better by peer review.

It trickles through in complaints about how long peer review is taking (when the review time may be reasonable), It trickles through when asking what reviewers could possibly say about a manuscript. It trickles through when questioning the value of journals organizing peer review.

This is a dangerous habit of mind for a scientist. As David Brin likes to say, “Criticism is the only known antidote to error.”

Sure, most scientists are professionals who are trained to produce competent science. It’s not surprising that most papers pass peer review, and that the improvement is not always that large. But there shouldn’t be an expectation that everything a scientist does is going to be worth publishing as is. Everyone makes mistakes.

As Rose Eveleth said yesterday:

Sure, some editors are annoying, but you know what is worse? Literally any writer’s raw copy.

This is a message a lot of scientists need to hear.

External links

How long should peer review take?

09 July 2018

Giving an article a DOI does not make it citable

With preprint servers and data repositories like Figshare gaining in popularity, a related myth is also gaining in popularity: “Giving this article / dataset a DOI makes it a citable academic product!”

Exposition for bystanders: DOI is short for “digital object identifier.” I usually describe it as “a serial number for a journal article.” You often seen them tucked at the top or bottom of academic papers,  a lengthy number beginning with 10. But as the name indicates, it’s mean to be usable for any kind of “digital object,” not just papers.

The beauty of a DOI is that if you have it, you can type in “https://doi.org/” followed by that number beginning with 10, and it will take you straight to the paper.No having to go to Google to find the journal, then drill down to the volume, then the issue, and so on.

A Twitter search shows people have been wrongly making the connection between “having a DOI” and “something that can be cited” for at least seven years. Here’s a tweet from 2011 (emphasis added):

#DataCite offers DOIs for making data sets citable and getting credits for its reuse.

A few example: here, here, and here. And also here, here, and here. And people have been pointing out this is wrong for the same amount of time.

What is “citable” is an editorial policy set by journals individually.

Most journals have long traditions of allowing you to cite things that are not part of the formal scientific literature. After all, journals existed well before links ever existed, never mind the DOI standard.

But not every journal will let you cite whatever. If a journal says, “We will only allow citations to peer-reviewed journal articles,” saying, “But this has a DOI!” is not going to make any editor change her or his mind.

I don’t quite understand why people think this. I suspect this myth arises because it plays into scientists’ obsession with formalism and simple “If this, then that” rules. Maybe people are confusing the DOI number itself with, “Anyone who goes to the bother of giving DOIs to things is probably an organization that is fairly large, stable, and has its act together.” But that exploitative journals regularly give their articles working DOIs shows that it can’t take that much to assign those number and get the links working.

It’s another example of the information vacuum around academic publishing: publishers make up new stuff and assume academics will figure out how it works.

While I’m here, Lyndon White pointed out another DOI myth: that the link “never breaks.” They can, although in practice I’ve found them to be far less susceptible to link rot than publisher links, which get rearranged every few years, it seems.

Related posts

Innovation must be accompanied by education
Why can’t I cite Mythbusters?

29 June 2018

Not hot: the Rate My Professor chili pepper is done

The website Rate My Professors is getting rid of its “hotness” rating. Which means you won’t see stuff like this any more:

I'll give you a chili on Rate My Professors if you give me an A

The idea of getting rid of the “chili pepper” has been floating around for a while, but fellow neuroscientist Beth Ann McLaughlin was able to hit a nerve on Twitter this week. Almost 3,000 retweets and many professors chimed in to say, “Get rid of this appearance rating.”

And to their credit, the website owners did.

This is a good thing for people in higher education. The Rate My Professors site is well known to people in higher education, both faculty and students. I’ve encouraged students to use Rate My Professors, because I have a record teaching, and people have a right to hear other students’ experiences. It matters when they tacitly suggest it’s okay to ogle professors.

It’s nice to have a little good news. And to be reminded that sometimes, faceless corporate websites – and the people behind them – do listen to reason, and can change.

External links

Why The Chili Pepper Needs To Go: Rape Culture And Rate My Professors (2016)
RateMyProfessors.com Is Dropping The "Hotness" Rating After Professors Called It Sexist
I Killed the Chili Pepper on Rate My Professor
RateMyProfessors.com Retires the Sexist and Uncomfortable “Chili Pepper” Rating After Academics Speak Out
RateMyProfessors Removes Hotness Rating