NIH Peer Review Advisory Committee Updates

The PRAC met earlier in November and covered a smörgåsbord of topics: electronic submissions, COI, EUREKA review, CSR realignments, new investigator initiatives, a Roadmap update, enhancing peer review update (focuses on the wonders of AED – asynchronous electronic discussions), review of F award applications, and more.

So … on the implementing peer review, something that caught my attention was the disappearance of the Priority 1 initiative to “compensate reviewers for their time and effort”, which would allow reviewers who have served a minimum of 18 full study section members (chartered member or equivalent) to apply for administrative supplements of up to $250K and/or request that they be considered for Merit/Javits on a competitive basis. This initiative appears on Kington’s third slide but disappears on the 4th where the timeline is laid out. Hmm. This would be a bummer to let go, and I hope they’re actually considering shortening the duration of service needed for this sort of bennie. The Enhancing Peer Review Website has been updated and will continue to post information on when planned initiatives will be implemented.

Nothing remarkable in the new investigator realm, though the PowerPoint does indicate that “Staff in the NIH institutes and centers will be apprised of ESI and New Investigator status and this factor will be considered when applications are selected for award.” Reviewers will also be told to take career stage into consideration. For those who hadn’t noticed this detail in the various announcements, the NIH will remove the New Investigator checkbox from the PHS398/SF424 forms and instead automatically identify new via their administrative database, while PIs will be responsible for updating their personal profile (degree/medical residency completion dates) in the eRA Commons to be considered for ESI status. Rockey’s presentation also indicates that “a procedure and guidelines for requesting an extension of the period of ESI eligibility will be developed and announced to accommodate individuals with various medical concerns, disability, pressing family care responsibilities, or active duty military service.” Update: The NIH issued a notice explaining how to update appropriate eRA Commons profiles to be considered for ESI eligibility and, if appropriate, request an extension of ESI eligibility.

Of particular interest was the insight provided into the EUREKA R01 review process, which differed slightly for the NIGMS versus neuroscience (NINDS, NIMH, NIDA) applications. Both used a structured review form for the initial review, which I recommend folks check out as a preview to the new structured summary statement format (though with different subheads and a 1-7 scoring option). Interestingly, NIGMS used AED for phase 2, while neuro used an in-person meeting at which 49 applications were discussed but only 28 scored (out of 140 applications). At NIGMS, 43 applications were discussed and scored out of 226 submitted. NIGMS funded 20, while the neuro cluster funded 18 applications.

And then there was the Roadmap update by Elizabeth Wilder, PhD, Director of the Division of Strategic Coordination at OPASI (though, see update in paragraph following), who talked about where all the Common Fund money is going (and reminded us that intramural NIH researchers compete alongside extramural applicants for these funds), changes to the application and review process for Pioneer and Junior Pioneer awards, and the Transformative R01 program (limited to extramural applicants). Of interest is her slide on the T-R01 applications that indicates the:

Primary required element would be statement of paradigm disruption/creation:

  • If a paradigm exists, why is it wrong and how will it be disrupted?
  • If no paradigm exists, how will project create one?
  • She then notes that the “review criteria will focus on transformative potential – ability of application to be paradigm disrupting, not just paradigm shifting.” Hmmm. Good luck, folks.

    One update not covered, at least in the PowerPoints, is the departure of Alan Krensky as Director of OPASI, as I noted previously, “to his laboratory in NCI and … Senior Advisor to the NIH Deputy Director” (now Larry Tabak, who also serves as Director of NIDCR). Although the official reports speak of reorganization and new efficiencies as the motivation, Science reports that “Krensky clashed with institute directors.” With regard to the OPASI change, briefly, Lana Skirboll, PhD has been appointed Acting Director of the Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI), which comprises OBSSR, ORWH, OAR, and ODP as well as the functions under the former OPASI (former Division of Strategic Coordination in OPASI will be reconstituted as the Office of Strategic Initiatives). I wonder who the new acting director of acronym assigment is.



    1. Thanks for the heads-up, dude! Going to study the PPTs now! Any extramural investigator who doesn’t pay attention to emanations from NIH like this shit is fucking stupid.

      Always a pleasure, Comrade PP. – writedit

    2. bikemonkey said

      that structured review form is, ummm, brief….

      something like two line comment fields?

      Comment fields will automatically expand as needed.”

    3. bikemonkey said

      oh ok, missed that…

    4. writedit said

      Letter in Science takes issue with various peer review reforms:

      Changes to NIH Grant System May Backfire
      Although we appreciate the many challenges facing the current NIH grant system and applaud attempts at reform, we believe that several of the changes planned (News of the Week, “Changes in peer review target young scientists, heavyweights,” J. Kaiser, 13 June, p. 1404) will cause more harm than good.

      The proposal to change the length of the R01 application from 25 to 12 pages, with a directive to emphasize potential impacts of the research rather than methodology, will have detrimental consequences for research. This extremely short application length will make it very difficult for reviewers to evaluate the qualifications of the investigator, the technical approach proposed, and the preliminary work that supports the proposed approach. It will be particularly difficult to evaluate team science, large high-throughput projects, and large interdisciplinary projects, including genomics projects, research resources, clinical trials, and bioinformatics databases that accelerate the work of entire scientific communities. Multidisciplinary projects will also suffer because of the inability to explain concepts from multiple disciplines to reviewers. Grant funding will hinge on the applicant’s ability to make flashy promises rather than to demonstrate technical excellence. We recommend a length limit of 20 pages, which would decrease the burdens of preparation and review, and represent a much smaller likelihood of hurting science. Another possibility would be to make application length proportional to budget requested.

      Forcing reviewer service for scientists who are principal investigators of three or more grants is also an issue. Three panels per year for 4 years (or even 6 years, as proposed), with a huge number of reviews on each panel, is too heavy a load. We propose requiring all investigators to serve on review panels if needed. This way, each investigator would review three times the number of grant proposals they submit (based on three reviews per submitted proposal), and would serve on no more than two panels per year.

      Another proposal that could have negative effects is the requirement that any investigator whose yearly funding would exceed $1 million provide reason for additional funds. It should be up to the review panel to decide whether an investigator can deliver great science given the technical merits of their application, their past track record, and the other demands on their time.

      This area is far too complex to be resolved by legislating a single cap or major barrier on funding, or by declaring that everyone must be equal. Considerations must be made for different investigator abilities and variability among scientific fields. Budgets should be defined by project needs, risk level, and potential payoffs, not by political limitations.

      Peter D. Karp (SRI International), Gavin Sherlock (Stanford), John A. Gerlt (University of Illinois, Urbana-Champaign), Ida Sim (UCSF), Ian Paulsen (Macquarie University – Australia), Patricia C. Babbitt (UCSF), Keith Laderoute (SRI International), Lawrence Hunter (University of Colorado), Paul Sternberg (Cal Tech), John Wooley (UCSD), Philip E. Bourne (UCSD)

    RSS feed for comments on this post · TrackBack URI

    Leave a Reply

    Fill in your details below or click an icon to log in: Logo

    You are commenting using your account. Log Out /  Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out /  Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out /  Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out /  Change )


    Connecting to %s

    %d bloggers like this: