At least one person was upset with this. During sessions, he asked, “If the program director makes these decisions, then what is the point of the panel?” I thought, “Oooh, green-eyed monster” when I heard that first question. Later, the same person asked, “So the decision is very subjective by the program director?” “It is, but it has to be very well justified,” was the reply.
At the end of the day, waiting for the shuttle, I overheard the same person again expressing his displeasure at the NSF. He said something like, “They are playing with people’s lives, and it’s all so subjective.”
Notwithstanding the undertones of entitlement in this person's comments ("They're giving my money to some researcher whose proposal was worse than mine!"), it seemed to me that this person’s worry was based on a false premise.
He presumed that a panel review can objectively measure the excellence of grant proposals. Indeed, I think he believed not only that the excellence can be measured, but that it can measured to several decimal places, like the roundness of electrons.
There is not that much certainty in evaluating proposals. In fact, there is research evidence that shows that there is a lot of wiggle in proposal evaluation.
I am glad that program directors have the ability to make decisions, rather than slavishly following a formula. While this obviously disappoints people who want to follow an algorithm (“I do X, then Y, revise my P and Q on the resubmission, ergo funding”), they ought to realize that there is no review process without a subjective element.
2 comments:
Whether it's a panel or one person, I agree that there's some subjectivity there. But wouldn't a panel be better at minimizing any evaluation bias?
The dynamics of group decisions are complicated. I think there is some evidence that committees often gravitate to extreme positions, not necessarily the “correct” ones.
NSF does uses a panel. The program directors pay very close attention to what the panel says. But any ability to override the panel recommendation seemed to bother this fellow, no matter if it resulted in (say) every award in a program doing to a single state.
Post a Comment