Proposal precision

At the National Science Foundation workshop I attended earlier this week, there was a lot of discussion about the power of the program director to award grants. As mentioned yesterday, a program director at the NSF doesn’t have go by the rankings of the panel reviewers. (This is rather different than NIH, apparently.)

At least one person was upset with this. During sessions, he asked, “If the program director makes these decisions, then what is the point of the panel?” I thought, “Oooh, green-eyed monster” when I heard that first question. Later, the same person asked, “So the decision is very subjective by the program director?” “It is, but it has to be very well justified,” was the reply.

At the end of the day, waiting for the shuttle, I overheard the same person again expressing his displeasure at the NSF. He said something like, “They are playing with people’s lives, and it’s all so subjective.”

Notwithstanding the undertones of entitlement in this person's comments ("They're giving my money to some researcher whose proposal was worse than mine!"), it seemed to me that this person’s worry was based on a false premise.

He presumed that a panel review can objectively measure the excellence of grant proposals. Indeed, I think he believed not only that the excellence can be measured, but that it can measured to several decimal places, like the roundness of electrons.

There is not that much certainty in evaluating proposals. In fact, there is research evidence that shows that there is a lot of wiggle in proposal evaluation.

I am glad that program directors have the ability to make decisions, rather than slavishly following a formula. While this obviously disappoints people who want to follow an algorithm (“I do X, then Y, revise my P and Q on the resubmission, ergo funding”), they ought to realize that there is no review process without a subjective element.