According to a news release today, CSR is sending invitations to about 2,500 members of the scientific community to become members of their new “College of CSR Reviewers,” which will review NIH applications in a process similar to journal reviews. College reviewers will review up to 12 applications a year for a 2-year period and may be eligible for the continuous submission policy. College reviewers typically will be asked to review one grant application in their area of expertise within 2 weeks, mainly as “mail-in” reviews as part of a 2-stage review process (as occurred with the Challenge Grants – mail-in reviews and editorial board review).
Stay tuned for more details and watch for that special invitation from CSR in your own in-box.
Medical Writing, Editing & Grantsmanship 
Comrade PhysioProf said
I wanna be Dean Wormer!!!
D said
No way. You are Bluto for sure.
Comrade PhysioProf said
No, fuck that! I call I get to be Dean Wormer!
D said
OK. Then I get to be Otter.
BB said
Ahem, sounds *almost* reasonable for a review system. 1 proposal a month for 2 years. And these are shorter proposals too. Yes?
writedit said
Yes, shorter proposals. I did not hear glad tidings about the editorial review process for the Challenge Grant applications, but then again, the mail-in reviews were, shall we say, challenged as well. Perhaps a thoughtful implementation of this approach would fare better, as Toni’s pilot study suggests (and according to their evaluation of the RC1 review process).
DrugMonkey said
Only 38% of the reviewers would choose that system for their own apps, though writedit. Think those numbers went up for challenge rounds? I don’t hear any praise from participating reviewers..
writedit said
Well, the full quote is even more cryptic. I assume the 64% refers to editorial board reviewers, while the 38% refers to mail-in reviewers (i.e., potential College members):
Toni views these data as evidence to press ahead with this approach though, and that’s what counts. As I mentioned above, editorial board members at the RC1 review meetings with whom I talked did not have positive views of the process. Mail-in reviewers commented more on the poor quality of applications themselves (& the chaos/rush involved in getting reviews in).
drugmonkey said
I thought perhaps it was all the first-round reviewers but parsed by the triage line…
BB said
PS: Forgot to wish happy holidays to WriteEdit.
writedit said
Thanks! Best wishes for success on research results and grant applications and kick-ass publications to everyone in 2010!!!
DrugMonkey said
Happy Holidays writedit!!
NAPS said
what a stupid idea this is. The crazy TS needs a CT scan!
D said
But think of the exclusivity. This is limited to only the first 2500 scientists who respond to their personal invitations! But wait there is more! Respond now and get a free Toni Scarpa signed 1.5ml tube opener! This collectible is worth $20 in the store. Get a friend to respond and we will double your order. That is two tube openers (a $40 value) absolutely free!
whimple said
Heckuva job Toni, heckuva job. 😛
NAPS said
Oh, I was not aware of them magic tube openers. Forget about what I said about TS and CT scan….new year wish list…me want to be on that exclusive list! Hee Haw.
a reviewer said
The “old” NIH system was consensus-based. People who were successful under that system liked it because it allowed them to get a feel for the tendencies of the study section through multiple submissions. People who were not successful under that system are, by and large, not doing science any more. The new system should result in more reviewer independence. This is a good thing.
These changes do not address the fundamental problem, which is the ability to discriminate among good to outstanding proposals. Based on my experience, the NIH system does not discriminate well among the top 20-25% of proposals, all of which have substantial merit. PI’s who consistently score around the 20th percentile are doing good science, but their proposals are not getting funded. PI’s who fare only slightly better do get funded. Then they go back to the well again and again with similar proposals and end up carrying multiple grants for closely related projects. This is a problem.
It might be better if each NIH program divided it’s budget among the top 20-25% of proposals, perhaps on a pro-rated basis so that the top 10% got a bit more. Under the current system, a proposal just below the “payline” might be fully funded, while one slightly above gets nothing, even though the difference in scores is not statistically significant.
The payline should be constant from year to year and not vary with the occupant of the White House and the whims of Congress. Individual grant budgets would have to be adjusted based on the overall NIH budget. But the goal should be to fund at least the top 20% of proposals, regardless.
writedit said
Nature Medicine has a bit more about the CSR College of Reviewers.
Payline Complexity Explained & CSR College of Reviewers Updated by NIAID « Medical Writing, Editing & Grantsmanship said
[…] also includes an update on the CSR College of Reviewers (discussed here previously), including the current membership roster. As a reminder, these folks (“editorial board […]