The use and misuse of journal Impact Factors continues to attract attention. I think Impact Factor has some usefulness in showing a journal is a bona fide scientific enterprise.
In my non-scientist life, though, I’ve seen other situations where people are aggressively pursuing some stat or another to prove their worth. When I was involved in the card game Legend of the Five Rings, there was a point where people were repeatedly calling for the game to use the Elo system to determine who were the best players.
Elo is not an acronym, but refers to Arpad Elo, who devised a system for ranking chess players. Like Impact Factor, it has sway. Players take Elo rankings pretty seriously for games where it is used. Many card gamers knew it from Magic: the Gathering.
I like this quote from Arpad Elo (written as part of a Chess Life article in 1962), because it showed Elo understood the problems of uncertainty well.
Often people who are not familiar with the nature and limitations of statistical methods tend to expect too much of the rating system. Ratings provide merely a comparison of performances, no more and no less. The measurement of the performance of an individual is always made relative to the performance of his competitors and both the performance of the player and of his opponents are subject to much the same random fluctuations. The measurement of the rating of an individual might well be compared with the measurement of the position of a cork bobbing up and down on the surface of agitated water with a yard stick tied to a rope and which is swaying in the wind.
Emphasis in the original.
Interestingly, after Legend of the Five Rings did adopt the Elo rating system, it did not prove to be “the answer” that some players thought it would be.
Picture of Elo from here.