11 July 2012

Success or spin? How to persuade a scientist

I spotted this on Facebook:

As Fry might say:

The problem with this is that are only two years shown.

In most social statistics, there is variation from year to year. Some years, a number will be up, and some years, numbers will be down. Someone who wants to create a particular image only has to look through all the years of data, pick a low number and a high number, and boom! You can suggest a trend when there is only random noise.

The data from the years 2001 through 2009 are needed to see if there is real trend here. A series of sparklines connecting the 2000 and 2010 numbers would help. Context matters.

Also note how some “net” numbers are shown as percentages of percentages. This can be misleading, particularly when you have small numbers. Small numbers are particularly susceptible to the fluctuations mentioned before, and are ripe for cherry picking. Going from 1% to 2% may well be a 100% increase, but it’s hardly worth trumpeting. It doesn't change that both numbers are still very small.
I hate to pick on something that has a positive message, but this is meant as constructive criticism.

1 comment:

Science Refinery said...

Lies, damned lies, and statistics, amirite?