Enhancing NIH Peer Review Cont’d

So, the good news is that the scientific community has an excellent representative on the NIH’s program Enhancing Peer Review in Keith Yamamoto, PhD (UCSF). You can see Keith, his intramural counterpart Larry Tabor (NIDCR), and the Great Zerhouni himself in a recent videoconferenced live discussion of the peer review reform initiative. The Powerpoint presentation to the research deans tunining in on this discussion are also available.

Issues raised after the PPT include blinded peer review, external hired reviewers, quality of peer reviewers (longing for the old days with smaller, higher quality panels reviewing fewer applications in more depth – the sort of study sections on which members looked forward to serving), role of publication record of reviewers, striking a balance between experts and generalists on panels, reflecting individual study section member discussion points, the need for special review panels for new investigators, the possibility of capping the number or dollar amount awarded to individual investigators (& possibily institutions), how to avoid bias based on submitting PI and/or institution, and the need to be able to recognize & reward promising new investigators through their track record &, well, promise. Some comments were a bit odd, such as why applications are dinged for environment (um, if they don’t have the resources and don’t collaborate with a group who does have the the appropriate facilities, how do they expect to do the proposed science?).

The Great Zerhouni noted that complaints about reviews being too conservative have persisted throughout the history of peer review and indicated this finding motivated the development of the Pioneer Award. The latter involves shorter applications and a 2-part review process (electronic and then interview) to reward truly transformative ideas, which has proved successful … per Elias. The GZ also confirmed they are considering a shorter pre-proposal submitted for review of innovation-significance of aims followed by invitations for full proposals. This makes excellent sense for the larger mechanisms in particular but could also weed out across the board the not-ready-for-primetime triage candidates with less pain and time investment on both sides.

You will learn that they will be conducting a series of 5 regional town hall meetings through October … that in Feb 2008, NIH “Leadership” will consider input from the RFI and the town hall meetings and the working groups and make suggestions for next steps, including possible pilot projects. Remarkably, the pilot projects will be designed and launched in March 2008 … the cynical among us would conclude that, well, the work on these pilots will be underway before all the data and comments are in and fully ruminated.

3 Comments »

  1. PhysioProf said

    “The GZ also confirmed they are considering a shorter pre-proposal submitted for review of innovation-significance of aims followed by invitations for full proposals. This makes excellent sense for the larger mechanisms in particular but could also weed out across the board the not-ready-for-primetime triage candidates with less pain and time investment on both sides.”

    As I have mentioned in comments over at DrugMonkey’s pad, getting detailed reviews on my first ever R01 application–which was triaged–was *exceedingly* useful in moving forward, both with my science and with the preparation and submission of a new application with very different specific aims.

    Nevertheless, it is undeniable that detailed review of clearly deficient applications takes time and effort that might be better spent on trying to figure out the difference between the 8th and 12th percentile grants. Maybe a good solution would be to ensure that all new investigator applications get full review–and even, perhaps, panel discussion–even if they are to be triaged score-wise.

  2. PhysioProf said

    One other thing: Writedit, if WordPress has this function, would you consider placing a “Recent Comments” section in your sidebar? It makes it a lot easier to follow, and promotes the occurrence of, ongoing discussion of your posts.

    Not ignoring your thoughtful suggestion, PhysioProf. Still need to work on a better color scheme for the deuteranopes out there, too. Just having trouble finding enough hours to sleep, never mind upgrade the look & feel of my blog. – writedit

  3. bikemonkey said

    you could also bookmark the comments feed, LazyProf!

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: