College of CSR Reviewers

According to a news release today, CSR is sending invitations to about 2,500 members of the scientific community to become members of their new “College of CSR Reviewers,” which will review NIH applications in a process similar to journal reviews. College reviewers will review up to 12 applications a year for a 2-year period and may be eligible for the continuous submission policy. College reviewers typically will be asked to review one grant application in their area of expertise within 2 weeks, mainly as “mail-in” reviews as part of a 2-stage review process (as occurred with the Challenge Grants – mail-in reviews and editorial board review).

Stay tuned for more details and watch for that special invitation from CSR in your own in-box.

19 Comments »

  1. I wanna be Dean Wormer!!!

  2. D said

    OK. Then I get to be Otter.

  3. BB said

    Ahem, sounds *almost* reasonable for a review system. 1 proposal a month for 2 years. And these are shorter proposals too. Yes?

    • writedit said

      Yes, shorter proposals. I did not hear glad tidings about the editorial review process for the Challenge Grant applications, but then again, the mail-in reviews were, shall we say, challenged as well. Perhaps a thoughtful implementation of this approach would fare better, as Toni’s pilot study suggests (and according to their evaluation of the RC1 review process).

      • DrugMonkey said

        Only 38% of the reviewers would choose that system for their own apps, though writedit. Think those numbers went up for challenge rounds? I don’t hear any praise from participating reviewers..

      • writedit said

        Well, the full quote is even more cryptic. I assume the 64% refers to editorial board reviewers, while the 38% refers to mail-in reviewers (i.e., potential College members):

        A majority of reviewers, some 64%, said they would choose this type of review for their own applications; of the experts who submitted written critiques considered by the editorial boards, about 38% said they would prefer these reviews for their own applications.

        Toni views these data as evidence to press ahead with this approach though, and that’s what counts. As I mentioned above, editorial board members at the RC1 review meetings with whom I talked did not have positive views of the process. Mail-in reviewers commented more on the poor quality of applications themselves (& the chaos/rush involved in getting reviews in).

      • drugmonkey said

        I thought perhaps it was all the first-round reviewers but parsed by the triage line…

  4. BB said

    PS: Forgot to wish happy holidays to WriteEdit.

    • writedit said

      Thanks! Best wishes for success on research results and grant applications and kick-ass publications to everyone in 2010!!!

  5. DrugMonkey said

    Happy Holidays writedit!!

  6. NAPS said

    what a stupid idea this is. The crazy TS needs a CT scan!

  7. D said

    But think of the exclusivity. This is limited to only the first 2500 scientists who respond to their personal invitations! But wait there is more! Respond now and get a free Toni Scarpa signed 1.5ml tube opener! This collectible is worth $20 in the store. Get a friend to respond and we will double your order. That is two tube openers (a $40 value) absolutely free!

  8. whimple said

    Heckuva job Toni, heckuva job.😛

  9. NAPS said

    Oh, I was not aware of them magic tube openers. Forget about what I said about TS and CT scan….new year wish list…me want to be on that exclusive list! Hee Haw.

  10. a reviewer said

    The “old” NIH system was consensus-based. People who were successful under that system liked it because it allowed them to get a feel for the tendencies of the study section through multiple submissions. People who were not successful under that system are, by and large, not doing science any more. The new system should result in more reviewer independence. This is a good thing.

    These changes do not address the fundamental problem, which is the ability to discriminate among good to outstanding proposals. Based on my experience, the NIH system does not discriminate well among the top 20-25% of proposals, all of which have substantial merit. PI’s who consistently score around the 20th percentile are doing good science, but their proposals are not getting funded. PI’s who fare only slightly better do get funded. Then they go back to the well again and again with similar proposals and end up carrying multiple grants for closely related projects. This is a problem.

    It might be better if each NIH program divided it’s budget among the top 20-25% of proposals, perhaps on a pro-rated basis so that the top 10% got a bit more. Under the current system, a proposal just below the “payline” might be fully funded, while one slightly above gets nothing, even though the difference in scores is not statistically significant.

    The payline should be constant from year to year and not vary with the occupant of the White House and the whims of Congress. Individual grant budgets would have to be adjusted based on the overall NIH budget. But the goal should be to fund at least the top 20% of proposals, regardless.

  11. writedit said

    Nature Medicine has a bit more about the CSR College of Reviewers.

    As Nature Medicine went to press, the CSR had asked close to 600 researchers to join the club, with roughly 80% of responders accepting the offer. The agency is now sending out around 100 invitations every few days, according to George Chacko, who heads the CSR’s bioengineering sciences and technologies integrated review group. The names of all of the College members are scheduled to be posted this month, the agency says.

  12. […] also includes an update on the CSR College of Reviewers (discussed here previously), including the current membership roster. As a reminder, these folks (“editorial board […]

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: