Medical Writing, Editing & Grantsmanship

Okay … a little help here, and this is no April Fool’s. I need to come up with 45 min worth of what not to do if you want to get funded. I could come up with dozens and dozens of bullets like this, and I have priceless gems from actual narratives that I couldn’t possibly make up if I tried. But I really am interested in what fatal flaws crop up in the routine grant-writing life of a reagent-quality scientist. When I’m all done in a few weeks, I promise to slap it up as a resource page (like NIH Paylines & Resources).

- Let the NIH assign your grant
- Mislabel tables/figures (better yet, don’t include any)
- Make legends small/cryptic (taking up valuable space)
- Don’t bother with letters of support/collaboration
- Only address the comments you agree with in your revision introduction
- Propose experiments using reagents/cell lines/transgenic models you don’t have & are challenging to generate
- Propose complex aims each of which is contingent on achievement of the one prior
- End vague sentences describing statistical analyses with “etc.”
- Mention by name as many well-known investigators as possible without explaining/justifying their role, which has no budget allocation either (no need for biosketch, everyone knows who they are - or a letter of support, hate to bother them)
- Use as many uncommon abbreviations as possible (defining each is optional - saves more space)
- Don’t waste space between paragraphs
- Remember, instructions are for sissies
- Never, ever show your grant application to anyone before submission

Two items in Nature journals consider the benefits of a consortium approach to peer review of journal manuscripts (in existence) and grant applications (modest proposal). Nature Neuroscience announced today it is joining the the Neuroscience Peer Review Consortium, which “reduces the overall reviewing workload of the community by allowing authors to continue the initial review process when their paper moves from one consortium journal to another, once the paper has been rejected or withdrawn from the first journal.” The Nature Neuroscience editorial describes the process, voluntariness, and flexibility and notes that the NPRC system will be evaluated at the end of the year … and on an ongoing basis at the journal’s blog, Action Potential.

Separately and quite distinctly, in a letter to Nature, Dr. Noam Harel of Yale makes a modest proposal: a centralized grant proposal repository into which applications could be deposited at the PI’s leisure and that sponsors could search for interesting science to review and possibly fund (no doubt with some encouragement by depositing PIs). The research proposals would only be made available to sponsor agencies, and multiple sponsors interested in the same work could collaborate on a shared funding agreement. As a thought exercise, interesting. As something to actually implement …

And finally, while we’re pondering peer review, Gregory Cuppan, a managing principal at McCulley/Cuppan (which specializes in document development), contributes to the commentary on a prior thread discussing the Publishing Research Consortium survey data. Specifically, he notes that “most people have little or no formal training in the task of review” and would “be interested to know how many readers of this blog have actual formal training in the task of review (here I make a strong distinction from training for the task of editing).” He refers to a 1961 study by the Educational Testing Service in which 53 distinguished reviewers read 300 college student papers but only had a median correlation among reviewer scores of 0.31.

In a separate note to me, he also suggested we look at an article by Mayo et al. in the Journal of Clinical Epidemiology suggesting traditional grant review processes and funding decisions suffer from a high degree of variability due to too few reviewers being involved. The report presents empirical data from intramural review of pilot project applications at McGill University Health Center Research Institute; applications were both ranked and scored (1-5 scale), with poor agreement between the two (kappa value of 0.36, with 95% CI 0.02-0.70). The top-ranked proposals would have failed to meet the “payline” with varying probability depending on who was assigned to provide a scored review. The examined process does not translate to current NIH study section practice, but it lends credence to the recommendation (see pp 4-5 & 38-41) that chartered study section members (not ad hoc reviewers) rank scored proposals at the end of each meeting. Per Mr. Cuppan’s suggestion that manuscript reviewers lack training, see pp 45-46 of the Enhancing Peer Review report for standardizing reviewer, chair, and administrator (officer) training.

Well, 18 months after relocating to Baby It’s Cold Outside, I’m still unpacking boxes & rifling through forests worth of files. Pay dirt! A few yellowing, crumpled mimeographs (okay, not that old & lacking that distinctive smell - but certainly photocopies of lists tapped out on manual typewriters) that I’ve managed to keep with me for nearly a quarter of a century, so heaven knows the original source. Suffice it to say, not me. The list of useful research phrases can be found online. I’m not sure I’ve seen this explanation for scientific method though:

The cornerstone of modern science is the scientific method. Scientists first formulate hypotheses, or predictions, and then perform experiments to test their hypotheses. There are two forms of scientific method, the inductive and the deductive.

INDUCTIVE:

DEDUCTIVE:

Today the Association of Research Libraries offered a Webcast (viewable via archive) on the recently mandated NIH Public Access Policy. The presentations by Kevin Smith, JD (Scholarly Communications Officer, Duke University), Jim Siedow (Vice Provost for Research, Duke University), and Tony Waldrop (Vice Chancellor for Research & Economic Development, UNC) were quite useful and raised awareness about the need for academic authors to protect and manage their copyrights more broadly. I think this is an hour well spent, especially for anyone actively involved in facilitating and monitoring institutional compliance with this directive.

Similarly helpful (especially to PIs without an hour to spare) and much appreciated are the resource links to materials from Duke and UNC, such as Duke’s sample letter to publishers to protect author rights to comply with the Public Access policy.

Finally, the NIH is seeking public comment on the Public Access policy. As with Enhancing Peer Review, written comments are due by Monday, March 17th. You can submit a comment to be presented at the Public Meeting to be held on Thursday, March 20th from 10 a.m. to 4:30 p.m. (NIH Natcher Conf Ctr); approximately 30-40 of these comments will be presented. If you want to attend, you must register by March 17th. The Great Zerhouni himself is on the agenda. The NIH has issued a formal Request for Information as well.

Update: Summary from the NIH Public Meeting: “The NIH received comments from representatives of universities and other NIH grantee organizations, publishers from commercial organizations and professional societies, journal editors, patients, public health advocates and the general public. The NIH received 451 comments in advance of the meeting. Preliminary analysis indicates more than 60% of these pre-meeting comments expressed support of the policy as implemented; approximately 15% said the 12-month delay period was too long; and 15% had concerns that a mandatory policy would be detrimental to scientific publishers.”

Nature Nanotechnology continues the discussion of journal peer review with a summary of and commentary on the recent Publishing Research Consortium report. Letters to Nature continue the discussion of double-blind peer review (pro, compromise, unworkable, need better data), as does the accompanying online discussion forum. [This same issue of Nature also includes some letters addressing duplication and plagiarism, and you all are encouraged to join the ongoing online debate on this matter.] Letters to Science bring suggestions for rewarding and rating journal reviewers (with, unfortunately, no linked discussion as is available at Nature).

And on the heels of intense discussion about the Dwyer et al. retraction, Science now publishes a “Letter of Concern” (a pre-retraction of sorts - “Science is publishing this Editorial Expression of Concern to alert our readers to the fact that serious questions have been raised about the validity of the findings in the Won et al. paper.”) the same week that Nature formally retracted one of its own reports when the authors “lost confidence in the reported conclusions … [and in turn] regret any adverse consequences that may have resulted from the paper’s publication.”.

First, an early casualty of Deja Vu as reported in Nature: “A review article written by a rheumatologist at Harvard Medical School in Boston, Massachusetts, has been retracted after the journal, Best Practice & Research Clinical Rheumatology, learned that more than half of the paper may have been plagiarized. The 2004 article, by Lee Simon (Best Pract. Res. Clin. Rheumatol. 18, 507–538; 2004), was manually checked after surfacing in an automated trawl through 7 million biomedical abstracts for possible plagiarism (see Nature 451, 397–399 ; 2008). The retraction was announced on 29 January. Harvard Medical School has formed a committee to review the matter but has not launched an official investigation, says spokesman David Cameron. Simon declined to comment, saying only: “I’m very sorry that I’ve been so targeted for something like a review article.””

This week’s Nature also includes a commentary on double-blind peer review of journal articles drawn from the Publishing Research Consortium survey results that “also highlight that 71% have confidence in double-blind peer review and that 56% prefer it to other forms of review. Support is highest with those who have experienced it (the humanities and social sciences) or where it is perceived to do the most good (among female authors). The least enthusiastic group is editors.”

Considering that so many manuscripts include referenced comments such as, “We previously showed …”, I have to wonder how blindable journal articles can be. Indeed, as Nature notes, “The editors at the Public Library of Science abandoned double-blind peer review because too few requested it and authors were too readily identified.”

Nature invites your comments on this commentary.

Most Current Discussion of this topic can be found here.

This week in Science, a retraction due not to misconduct [see discussion below - an investigation could follow] but to the community of science working as it should:

We wish to retract our Report “Computational design of a biologically active enzyme” [MA Dwyer, LL Looger, HW Hellinga, Science 304, 1967 (2004)], which describes triose phosphate isomerase activity in a computationally redesigned ribose-binding protein (RBP) from E. coli. Dr. John P. Richard (Dept of Chemistry, Dept of Biochemistry, SUNY Buffalo), to whom we provided clones encoding the novoTIM activity, has brought to our attention that the triose phosphate isomerase activity observed in our reported preparations can be attributed to a wild-type TIM impurity–seen in preparations that use a continuous rather than stepwise imidazole gradient (as in the original paper) or that add a second sepharose column. Richard’s reanalysis has now also been confirmed by others in the Hellinga laboratory. The interpretations in the original report were based on lack of observed activity in mutant, engineered enzyme that bound substrate, but lacked catalytic residues. Variations in expression levels of designed proteins relative to the amount of contaminating endogenous protein might account for the pattern of observed activities that led to our erroneous conclusions. The in vivo experiments have not been reexamined.

We deeply regret that our report of a designed enzyme activity does not live up to closer scrutiny. Nevertheless, we remain optimistic that the problem of structure-based design of enzyme activity will be solved and that novel catalysts will be produced in conjunction with computationally based methods.

Mary A. Dwyer, Dept of Pharmacology and Cancer Biology, Duke Univ Med Ctr
Loren L. Looger, HHMI, Janelia Farm
Homme W. Hellinga, Dept of Biochemistry, Duke Univ Med Ctr

Update: We already have an article retraction resulting from use of this database, and both Deja Vu and the underlying tool, eBlast, were mentioned (though apparently not used) in conjunction with a massive case of fraud (70 articles published over a 4-year period retracted).

In Nature this week, we have a Tale of Two Citations. A tale with which most of us are all too familiar. Best of times and worst of times and all that. A tale, um, retold in The Chronicle for Higher Education (though as a report about the Nature commentary).

In a nutshell: “Although duplicate publication and plagiarism are often discussed, it seems that discussion is not enough. Two important contributing factors are the level of confusion over acceptable publishing behaviour and the perception that there is a high likelihood of escaping detection. The lack of clear standards for what level of text and figure re-use is appropriate (for example in the introduction and methods) is a well known problem; but the belief that one can get away with re-use is probably the single most important factor.”

In addressing the latter issue, the authors, Mounir Errami and Harold Garner (both from University of Texas Southwestern Medical Center), used the search engine eTBLAST to look for similar language among papers and found 70,000 hits in Medline. These and duplicate citations from other scientific literature databases are deposited in Deja Vu. The authors have manually read through 2,600 abstracts from these hits and would like help from the scientific community in sorting through this repository. Dr. Garner has received a Research on Research Integrity R01 award to support this work.

In the Nature article, the authors suggest that if journal editors “use more frequently the new computational tools to detect incidents of duplicate publication — and advertise that they will do so — much of the problem is likely to take care of itself.” Perhaps, especially as manuscripts were returned and word spread along the grapevine.

Unfortunately, one possible source of high duplicate hit rate could be the increasing tendency for e-pub ahead of print, which could account for identical (or near identical) abstracts and authors within months of each other in the same publication. However, this would not be a problem for journal editors considering a newly submitted manuscript. Perhaps the International Committee of Medical Journal Editors would consider addressing this possibility (use by journal editors of electronic tools to flag plagiarism, self or otherwise) in a future update of their guidelines. Perhaps the folks involved with the NIH Public Access Policy and PubMedCentral might consider taking a look-see at their own repository.

I had to cobble something together for the health sciences faculty here (an intimate group of a few thousand of my closest friends) on how the NIH Public Access Policy actually works, and I thought I’d share the wealth here. Hope this is helpful.

All peer-reviewed articles that arise from work funded in part or in whole by grants or contracts awarded by the NIH must be made publicly available no later than 12 months after the official date of publication.

This mandatory requirement applies to all grant and cooperative agreements awarded on or after October 1, 2007 and to all contracts awarded on or after April 7, 2008. Principal Investigators are responsible for ensuring compliance with the Policy even if they are not an author or co-author of a publication arising from their NIH-funded work.

Compliance is a three-step process: address copyright when submitting a manuscript to a journal for review, submit the accepted manuscript to the NIH, and cite the manuscript using the PubMed Central reference number.

Address Copyright. As an NIH-funded author, you must ensure that any copyright transfer or other publication agreements allow the article to be submitted to NIH in accordance with the Policy. If you submit your manuscript to a journal that already cooperates with the NIH on this, you need not do anything further to ensure compliance with the Policy.

If you wish to publish in a journal not on this list, you must inform the journal that the article is subject to the Public Access Policy when submitting the manuscript. You must then communicate with the journal before any of your copyrights are transferred to ensure that all conditions of the Public Access Policy can be met. Some journals ask authors to transfer copyrights when a manuscript is submitted for review, so please be careful not to transfer rights during the submission process without confirming the journal’s ability to comply with the Public Access Policy. Please remember that this Policy is mandatory for all NIH grantees, and avoid signing any agreements with publishers that do not allow authors to comply.

Submit Manuscript to NIH. If your journal does not submit the accepted manuscript to the NIH on your behalf, you or your designee can deposit a copy of the peer-reviewed manuscript via the online Manuscript Submission System (tutorial available). The submitted final, peer-reviewed manuscript must include all graphics and supplemental materials that are associated with the article. Please note that not all open-access journals automatically submit manuscripts to the NIH; you are responsible for ensuring your manuscripts are deposited in compliance with the Policy.

Cite PubMed Central Reference Number. Beginning with the May 25, 2008 receipt date, your applications, proposals, and progress reports sent to the NIH must include the PubMed Central (PMCID) or, if the PMCID is not yet available, the NIH Manuscript Submission reference number (NIHMS ID) when citing applicable articles that arise from your NIH-funded research.

You’ll find additional information and details on how to comply with the Policy at:

NIH Public Access Policy Notice

NIH Public Access Policy

NIH Manuscript Submission System

PubMed Central

But first … an NYT Magazine feature on The Moral Instinct. And in February, check out The Scientist for a discussion by C. Neal Stewart Jr. and J. Lannett Edwards on their approach to teaching a successful research ethics class. In the meantime, some additional perspective by Edyta Zielinska.

Gradually, the data are coming in regarding whether required RCR training and/or real-world ethics “lessons” affect the behavior of researchers. Most recently, McGee et al. offer results from a qualitative study of doctoral students and postdocs before and after a formal RCR course at the Mayo Clinic. Interestingly, none of the authors are from the Mayo.

Interviews were conducted ($40 compensation, but only 30 of 127 class matriculants participated in the study) to learn whether prior experiences influence how a scientist reacts to and is or is not influenced by a formal (weekly lecture) RCR course. Of the 30 study subjects, 13 reported having already taken a formal RCR course (in one interview, a participant says he/she has “done this … five or six times”!), and 15 had “informal” RCR training (not defined, but I assume via mentor, perhaps workshops or seminars or even online modules … this irritates me because I’ve seen “informal RCR training” used elsewhere without being well defined).

The authors opted to focus on two lectures based on results of a small pilot study: Authorship (RCR students expressed strong opinions & frank disagreement about material taught) and Conflict of Interest (RCR students had little prior knowledge & displayed much confusion about this concept). At the Mayo, 2 different instructors presented these lectures. Read the rest of this entry »