Comments on: Pooling Peer Review

By: Direct Public Funding of Pilot Research « Medical Writing, Editing & Grantsmanship

Sun, 19 Oct 2008 13:48:05 +0000

[...] would consider a central proposal respository such as that proposed by Noam Harel at Yale (as we discussed earlier). This way, the public could sort through what’s available and, if inspired, submit an [...]

By: Noah Gray

Noah Gray — Wed, 09 Apr 2008 18:07:24 +0000

With some modifications, there is a chance that this proposal could work. But I have my doubts, not that it is necessary or could be useful, but because of my interpretation of the scientific community. Feel free to view all of my concerns here. Thanks, Noah. I need to sort through your thoughtful coverage of this concept, but in the meantime, thanks too for the link to a great New Yorker cartoon on observations the impact of grant funding. Another favorite cartoon on not funding grants is there, but not my all-time favorite grant-related New Yorker cartoon. Damn. - writedit

By: drugmonkey

drugmonkey — Fri, 04 Apr 2008 04:06:06 +0000

Now surely you are not equating my humble contributions with The Scientist! We don’t have that big an impact, geez.

Noah’s got something up over at Action Potential….I guess I’d better take a whack at it…

By: Noam Y. Harel

Noam Y. Harel — Sun, 30 Mar 2008 19:43:57 +0000

Hello,

BB said:
“Two things come to mind. First, Noam Harel’s idea is already in place by several grant consortia. One of the, the Gotham group, I looked at last year. One problem I have with grant consortia is the possible lack of confidentiality of one’s ideas. Because of that, I never submitted to the Gotham group in the end.”

The Gotham Prize Foundation is one of my inspirations for suggesting a Centralized Proposal Repository (CPR)- I duly credit it on my website. However, that is just one funding source and one (broad) topic. True, their proposals are open to the web for review and comment (and collaboration) – however, the CPR does not need to be any less confidential than the current grant submission systems already in place. It will merely expose each proposal to a larger number of agencies and potential funding. It’s akin to a universal application for college or med school/grad school – it would make life a lot easier.

Please see a more detailed version of the CPR proposal on my website at http://noamyharel.googlepages.com/universalproposalrepository

Thanks! Noam

Cool. Thanks for alerting us to your CPR proposal Website. I’m sure if DM picks this up over at Scienceblogs and/or The Scientist runs with the idea, you’ll get the scientific community buzzing. I hope you submitted these ideas during the Enhancing Peer Review comment period. I need to read through the full CPR proposal still, but do like the concept – and maybe the federal agencies can break down enough silos, especially with the advent of grants.gov, to make this workable. I think RFAs and PASs (& some PARs) would still require targeted separate applications, but the IC PAs and parent PAs could simply serve as guides for research priorities as they currently do. – writedit

By: Marilyn Mann

Marilyn Mann — Fri, 28 Mar 2008 18:52:18 +0000

Hi everybody. I have a question. I was reading an article from Archives of Medical Science. I noticed that the abstract appeared to have an error in it. A reference to “LDL” should have said “HDL” (based on the relevant table and discussion in the article). I sent an email to the editor of the journal noting the error. He sent me a reply noting that there was still time to write a letter to the editor. I sent two emails to the corresponding author and did not receive a reply.

I’m a lawyer, so I’m having trouble understanding the correct procedure here. Is it really necessary to write a letter to the editor to point out this kind of error? Seems like overkill. Why doesn’t the journal just make the correction?

Thanks, Marilyn

I have to admit, I am not familiar with the Archives of Medical Science. How the editor handles this stuff is his or her call. Most would print a correction in the next issue (plus put an online correction immediately). Corrections are up to the editor rather than the corresponding author, and one would hope he/she would have noticed the error and reported it to the journal as well. It seems this editor prefers to give you “credit” for correcting this error, hence the request for a letter to the editor. You’re right – it is odd. You’ve reported the error, and the editor (upon confirming the correction with the author) should change the electronic version of the article whether or not you submit a formal letter.

By: BB

BB — Fri, 28 Mar 2008 15:07:46 +0000

No, no, bikemonkey, not at all.
We hates them, to quote Gollum. Trying to point out (and not succeeeding, I see) that we come out of grad school ill-prepared to write grants well, or at least, to write grants to satisfy study sections.

By: bikemonkey

bikemonkey — Thu, 27 Mar 2008 21:58:24 +0000

But grants are different. Factoring in esoteric qualities like previous grant history, which member of congress is leaning for funding for a pet project, grant horse-trading, the old boys network, multiplies the complexities of the grant review process. Science might be the least of it in too many cases. BB you sound suspiciously as though you are actually endorsing these factors as legitimate??????

By: maxine

maxine — Thu, 27 Mar 2008 18:22:12 +0000

Nice post. There is a formalised peer-review training process at the BMJ, as described here (free access) by Trish Groves, deputy editor of the journal http://blogs.nature.com/peer-to-peer/2006/06/quality_and_value_how_can_we_g.html
and at the journal’s website.

I like the point about matching articles with appropriate reviewers, which is a serious problem given the escalating number and diversity of grant applications versus the limited number of study section members plus limits on what the SRO knows about each reviewer’s range & depth of expertise (standing member or ad hoc invitee). Thanks for the link and the note about this process at BMJ, Maxine. -writedit

By: BB

BB — Thu, 27 Mar 2008 17:38:49 +0000

Two things come to mind. First, Noam Harel’s idea is already in place by several grant consortia. One of the, the Gotham group, I looked at last year. One problem I have with grant consortia is the possible lack of confidentiality of one’s ideas. Because of that, I never submitted to the Gotham group in the end.
The second comment I have is about training to review. I think a lot of us trained to read papers critically, through journal club. We know how to review a manuscript. But grants are different. Factoring in esoteric qualities like previous grant history, which member of congress is leaning for funding for a pet project, grant horse-trading, the old boys network, multiplies the complexities of the grant review process. Science might be the least of it in too many cases.
At a grant writing course I went to 5 years ago, we had mock study section. Maybe this is a way to train people for grant review. It certainly helped me focus my grant writing a lot!

BB, I like the Journal Club point a lot … and research trainees in good labs should get a similar experience at weekly lab meetings where grant proposals in progress are discussed. (I hope this concept sounds familiar to at least a few folks out there.) Baby It’s Cold Outside has several for-credit courses in various Schools & departments in which grant applications are prepared over the course of a semester, with all students participating in the review/critique of each other’s narratives. Maybe one of these days I’ll try to write out what it is I do when I take apart and critique a grant application I’m asked to review. As I told DM, I blame this odd gift on a mild concussive injury on the rugby field. – writedit

By: bikemonkey

bikemonkey — Thu, 27 Mar 2008 17:11:21 +0000

if by “members of this august community” you mean the applicants and peer members of NIH study sections, hell yes many of us what these issues explicated.