NIH Reviews, Reviewers … and new Director

In Science this week, Sudhansu K. Dey, PhD, suggests that the NIH Needs a Makeover to rectify a grant-making process that “has become tangled and inefficient”, mainly due to the lack of quality reviewers. Dey is not speaking as a jilted or jaded applicant, as his impressive NIH funding record demonstrates. Indeed, his concern seems to lie in the fact that his peers (in terms of funding-publications) are not the ones raising their hands to review applications, leaving SROs scrambling to fill study sections:

The Center for Scientific Review is desperate to recruit reviewers and is drafting individuals who have poor records of NIH grant awards or weak publishing histories. How can those individuals be trusted to review grants?

He suggests CSR make more use of videoconferencing to entice participation by busy established investigators, something Toni Scarpa favors as well. However, some reviewers have told me this works better for smaller groups discussing a manageable number of focused applications than for full chartered study sections covering a wide swath of science and funding mechanisms … and that they would prefer not to give up the face-to-face exchange – both to rigorously discuss applications and to network with peers.

Dey also briefly touches on the new scoring and critique system, suggesting he himself is not clear about whether the scores for individual review criteria “have any bearing on the overall score.” Having now seen a variety of summary statements from both discussed and triaged applications for different ICs and mechanisms, I myself wonder – upon reading the bulleted comments in relation to the scores given (criteria & impact) – how well these sync up in terms of the message the reviewers intend to convey. As I noted previously, CSR has a wealth of data to mine in the 20K RC1 reviews, at least in terms of whether the criteria scores match the comments made and send unambiguous feedback to the applicant.

Finally, Dey shares his Makeover advice on the next NIH Director, likely before Francis Collins’ appointment and rapid confirmation:

It is time to appoint a strong leader at NIH who has the understanding of a lifetime researcher and the authority to revolutionize the institution.

Collins is a “lifetime” researcher, perhaps, but one who has spent most of his career within rather than outside the NIH. Indeed, perhaps not surprisingly, Collins does not feel the intramural system is in need of change:

Asked about NIH’s intramural program, he said he is “resistant to the idea that [the program] is in need of some sort of dramatic redo” but is pondering whether to create a pool of intramural money that, like NIH’s Common Fund, could be used to fund crosscutting research quickly.

I suspect Dey was thinking in terms of a “lifetime” researcher who has weathered the turbulence of extramural funding and understands the reality of maintaining a lab dependent on the same. As an aside, Neil Greenspan, MD, PhD, commenting in The Scientist, wonders if Collins is out of touch with reality in a different regard, namely, the realistic potential of genomics in clinical practice. Both of these concerns are dwarfed, however, by the importance of the NIH Director’s ability to get Congress to cough up more money for the base appropriation. Given the current economic climate, Collins has a tough row to hoe here as well.

15 Comments »

  1. Man, that dude is such a fucking whiner. I find it hilarious that scientists completely jettison the scientific method when they bash the NIH peer review system.

    • bikemonkey said

      Word. More of the usual unsubstantiated whinging and data free asserting. Just look for a testable hypothesis…

  2. whimple said

    The Center for Scientific Review is desperate to recruit reviewers and is drafting individuals who have poor records of NIH grant awards or weak publishing histories. How can those individuals be trusted to review grants?

    By “be trusted to review grants” he obviously means, “be trusted to maintain the status quo in grant review”. We could simplify his life and peer review generally by having him and other trusted scientists (presumably he has an idea of who these people would be) come up with a list of people who always do quality work. These individuals could then have their grants approved after a simply pro-forma programmatic review, rather than cluttering up the study section review process with grants that are destined to be funded anyway.

  3. Luna Halloween said

    Indeed, Dr Dey is a real whiner.

    Dr Dey needs a well-planned makeover. His history of funding is absolutely impressive. His research is on a very important biological subject, preimplantation. Preimplantation is also a very important medical problem. But his scientific trajectory is not impressive at all in spite of his high numbers of papers.

  4. Danny O'Rerio said

    All NIH needs to do to solve their ‘reviewer problem’ is assign full membership in a study section with each R01. They’ll then have plenty of participants, who also happen to be the most successful applicants. Since publication rate and funding success are presumably correlated, they’ll also be getting reviewers with good publication records.

    It’d be like getting a military reserve scholarship. They pay you, but then you have to put in your time.

  5. Luna Halloween said

    I am not sure that publication rate: funding success’s correlation is the most reliable index to ensure innovative-competitive scientific research, according to recent history in the USA.

    Dr Dey claims:

    “The Center for Scientific Review is desperate to recruit reviewers and is drafting individuals who have POOR RECORDS of NIH grant awards or WEAK PUBLISHING histories. How can those individuals be trusted to review grants?.”

    I have no idea if the CSR is desperate or not to recruit reviewers but Dr Dey is the author of an article in Science: NIH needs a makeover.

    Dr Dey is an author in 364 papers (Pubmed). In one of the papers, published in 2000 in PNAS:

    “Coordinated regulation of fetal and maternal prostaglandins directs successful birth and postnatal adaptation in the mouse”

    Dr Dey claims “Parturition failure in COX-1−/− females (8, 9) and reduced uterine contractions in response to NSAIDs (18, 24–26) suggest that PGs are required for parturition”.

    Well, that “PGs are required for parturition” in humans is pretty well substantiated and is far from being a recent finding. I regret that the reviewers of this paper did not alert Dr Dey on that and some other claims of the paper. So, Dr Dey got a quick pass to this paper in a top journal (PNAS), perhaps because of his name and great rate of publication. And perhaps, for the same reason, he has been getting a quick pass in funding.

    The CSR should look for reviewers behaving in a more rigorous fashion, making sure that truly innovative research has a chance to get funded. This is the makeover that, in my view, CSR/NIH needs.

  6. Pinus said

    I am not defending anything this guy says…but if all you can get on him is a not so interesting PNAS paper, then he must be golden. PNAS is well known for crony publishing…you know the whole ‘academy members’ get to publish their stuff there thing.

  7. Luna Halloween said

    I would not restrict the “crony publishing” to just PNAS. The same would apply to Science, Nature and some others “leading” “club members”.

  8. Pinus said

    Luna,

    I agree that there is a fair amount of ‘cronyism’ that occurs at other big journals. My point is that it is a publicly stated policy at PNAS.

  9. D said

    I wonder if this is why he wrote the letter to Science?

    According to NIH Reporter (http://projectreporter.nih.gov/reporter.cfm), although he has two active Merit Awards (r37s) One of them was last funded in 2008. Which means that it didn’t get renewed in FY 2009. It is on a no cost extension. Last chance is Sept. 2009 Council.

    So, he is ticked-off because a 30 year old R01/R37 grant might get dumped. Of course it has to be the lack of quality reviewers not quality science in the app.

    Perhaps if he made his SS available we could all judge for ourselves.

    • Luna Halloween said

      D,

      He should not have to make his Summary Statement (SS) available.
      Summary Statements should be posted in NIH Reporter as justification of an award. I mean, why the scientific community (reviewers committee) believes that a specific research proposal has public health relevance and, therefore, deserves public funds?. (see Summary Statement). This would also be an ongoing educational instrument for the rest of scientists who have not participated in the review, as well as for the public. It may also help, along the way, to establish correctional factors if there is sufficient evidence of dissonance between proposed goals and attained benefits for science (e.g., in terms of innovation or advancing a field) and public health.

      A concept that I never understood in the NIH policies (reasons to appeal a review) is that” differences in scientific opinion are not considered a reason for appeal”.

      Why not ?. Why not giving a chance to a researcher whose proposal presents a different view on a scientific or clinical question?. Is it not what science is all about ?. Do differences in scientific opinion have a base for potential innovation?.

      Sorry D for the diversion from your point. It just came to my mind.

  10. D said

    Luna,
    The SSs I want to see are for those apps that are not funded. NIH won’t even acknowledge receiving an app unless it is funded. The simple announcement of the number of RC1 apps received and triaged is a big change.

    I believe that if someone wants to bitch about their bad review, especially publicly in a major journal, they should put up (the SS) or shut up.

    Also, if you were allowed to appeal based on differences of scientific opinion Councils would be re-reviewing 80-90% of all applications. That is not feasible.

    • Luna Halloween said

      Thanks D.

      Very good points. I appreciate it

      • D said

        De Nada. And I must say that I am jealous of your nom de plume. Mine is soooooooo borrrriiinnngg.

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: