NIH Box Score

Direct from Tony “You Gotta Problem With Dat Priority Score?” Scarpa … for FY 2006:

– 80,000 applications received
– 55,000 applications reviewed
– 18,000 reviewers
– 238 Scientific Review Administrators
– 1,800 review meetings

And for comparison ….

-80,000 applications in 2006 vs 46,000 applications 5 years ago
-39,000 R01s & R21s reviewed by CSR in 2006 vs 21,000 of the same 5 years ago

and finally …

In May 2006, 14% of new investigators and 18% of established investigators received a new R01 (Type 1) score from CSR within the 20th percentile, with 34% of established investigators scoring in this range with their competing renewal (Type 2).

5 Comments »

  1. drugmonkey said

    The typical lying by misdirection that I detest. Who cares a rip about 20th percentile when the funding line is 10%ile. How many New / established in the 10%ile range Dr. Scarpa? That’s what matters. You’ll see similar bogus stats with scores- Scarpa likes to say that New Investigators score better. Leaving aside the issue of the importance of a given application to someone who has no funding versus someone who does, what does it matter if New investigators average, say, a 190 and established average a 210? The funding line is down at 140!!!!

    Ask the hard questions at these official NIH presentations people, it is the only way they are going to get it…

    To be fair to Dr. Tony, CSR has many years of data for comparative purposes that use the 20th percentile as the cut-off – hence the continued use of the same. Also, grants up to the 20th percentile can get funded. Program officers can and do select grants up to the 20th percentile (or an equivalent distance above the payline for priority scored applications) and request that the IC Director consider them for funding. I’ve seen it happen many times – usually because the PI has worked closely with the program officer and convinced him or her of the project’s importance to the IC’s programmatic goals. Just as reviewers champion specific applications in study section (yes, I know that happens at every review session), program officers champion specific applications in the Advisory Council and the Director’s office. The proposal needs to have scored reasonably well to justify this effort, but it is quite common and a clearly stated policy in all the “funding strategies” so PIs don’t get pissed off when their 7th percentile submission isn’t funded (because the IC is already funding 10 excellent projects addressing the same theme) but a novel 19th percentile application filling a needed gap in the portfolio is awarded. (your program officer can confirm whether your proposal is filling a needed gap – just ask) In these times of uncertain funding and desperate competing renewals, this going outside the payline/percentile cut-off is more and more frequent – thanks to the personal relationship PIs maintain with their program officer. And believe me, the NIH folks are depressed about the current status … they aren’t ducking the hard questions – they have no answers, but I do believe they are doing their best with the crumbs they receive. – writedit

  2. drugmonkey said

    i know all of this, trust me. the use of the 20%ile may be for historical reasons but at the moment it seems out of touch at best and a pernicious attempt to underplay the plight of the New Investigator at worst. I favor the latter interpretation because they also pound the “better scores” drum to which I alluded before.

    Yes, Program can pick up New Investigator apps that fall outside the funding line, and they do. My point is that they shouldn’t have to do so. My contention is that there still remains a huge set of biases against the transitioning investigator at the level of primary review and that this is what really matters to outcome. Instead of trying to “fix” the outcome after the fact I think the NIH should focus on “fixing” study sections. Interestingly, it is in vogue to complain about bad review but everyone seems to feel that what is wrong is that study sections are not biased *enough* for the established investigator with a bad proposal…

    how is the “best possible science” funded through who happens to be BFF with a Program officer?

    Well, with the funding situation, scoring and award decisions will continue to be strained, though I’m not as jaded as you are about the process, Drugmonkey. Overall, the system works. Now, for the new PI in particular, I cannot tell you how many proposals from junior faculty/postdocs I have reviewed that display astonishing ignorance of the scientific method and have no business being submitted much less funded. Without mentors to genuinely guide hypothesis development and study design, these scientist wannabes cannot be expected to prepare competitive proposals in the fat times, much less the very lean times as we are experiencing now. – writedit

  3. drugmonkey said

    I tend to be vocal about the plight of the New Investigator, true. But it is soooo easy to intentionally misunderstand that I somehow think that New Investigators should be funded across the board. Not so. I’m not bemoaning the fate of *anyone* who can’t write a proper grant. No worries there. If a New Investigator is learning the hard way how to prepare a proposal, so be it. Moving from triage to 230 is not my concern.

    My concern is the fate of the *excellent* New Investigator application and whether such an application is being treated fairly against applications from established investigators. My section sees plenty of fantastic apps from New Investigators. Everybody talks the right talk about them- “highly enthusiastic about this very promising young scientist with this great new approach”. My problem is when that enthusiasm results in a 180 score, which is common. I then become incensed when the proposal that “display(s) astonishing ignorance of the scientific method” from Professor Bluehair gets roundly bashed for being obscure and confusing yet ends up with a 140 on the basis of “well I know from her great pubs that fantastic science will result regardless”.

    how do you know “the system works”? how do you know it might not work better some other way? if it “works” why are we being subjected to so much change, from roadmap to “emphasis on significance” to New Investigator brow beating?

  4. Harpa combined said

    “your program officer can confirm whether your proposal is filling a needed gap”

    What a joke! None of the great Program Officer him or herself was EVER a grant recipient. They got their jobs through backdoor—-and you expect them to tell you whether your grant is good or bad?! The reform is a way these public servants find something to do, to tell taxpayers that “we are doing something, we have no faults”. Wait and see after a few years, they will show you some charts and data and start another great reform. Their leaders will travel every campus to “explain” to you how you will be benefit from this reform.

  5. Harpa combined said

    The writedit guy even tells you to “schmooth” with the program officer to get funding—if you don’t do this— too bad, no money for you. As they all come through the backdoor for their positions, they in turn feel “kiss my ass to get what you want”—as they kissed to get their jobs. Can Mr.writedit put a chart up with how many PO or SRO ever being a grant awardee? How many used to work in NIH as failed postdoc or PI but with personal connection to get their job? Isn’t this the biggest public corruption when these people control the funding and the fate of America’s science?!

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: