Medical Writing, Editing & Grantsmanship

Update: As noted below in Brendan’s comment , open discussion of this issue continues at Nature Network. Please join in there! The Chronicle is also having a lively discussion of “the oath” … and JAMA has a nice summary of why even well-intentioned folks cannot overcome unconscious bias caused by conflicts of interest, and The Lancet notes this commentary in covering a plagiarism case in the UK. Links to The Gallup Organization report on which this commentary is based can be found at ORI and below.

Nature reports a survey conducted by the Office of Research Integrity that, not surprisingly, finds that most misconduct goes unreported. Sandra Titus et al. found that the “2,212 researchers we surveyed observed 201 instances of likely misconduct over a three-year period. That’s 3 incidents per 100 researchers per year.” (an average of only 24 institutional investigation reports are submitted to ORI each year).

Titus et al. contacted 4,298 scientists holding NIH extramural research funds at 605 institutions.

In 2006, we asked participants to indicate the number of times they had observed suspected research misconduct in their own department in the past three academic years (2002–05). 2,212 scientists provided complete responses to questions concerning research misconduct (51% response rate). Of these, 192 scientists (8.7% ) indicated that they had observed or had direct evidence of researchers in their own department committing one or more incidents of suspected research misconduct over the past three academic years. The 192 scientists described a total of 265 incidents.

Scientists were asked to indicate how they became aware of the possible misconduct and were told to report observations and not hearsay. … We used these descriptions to validate whether the observation met the federal definition of research misconduct [fabrication, falsification, plagiarism].

Two people independently coded and evaluated the 265 descriptions to determine whether each met the federal definition of research misconduct. In all, 64 reports (24% of the total) did not meet the threshold of the federal definition — which left 201 observations of potential misconduct made by 164 scientists (7.4% ). These 201 misconduct observations included fabrication or falsification (60% ) and plagiarism only (36% ).

According to our respondents, 58% of the observed incidents had been reported to officials at their institutions.

The authors go on to extrapolate that annually “approximately 1,350 would have been reported [compared with actual number of 24] whereas almost 1,000 could be assumed to go unreported to any official.”

As a fan of work on procedural and distributive justice by Brian Martinson, Melissa Anderson, et al., I was particularly pleased to see the accompanying editorial, Solutions - not scapegoats, address the need to examine the environment as well as the individual:

Meanwhile, misconduct investigations all too often focus solely on an individual offender, and fail to diagnose the environment that has allowed misconduct to flourish. Instead, institutions should seize the opportunity to learn from the experience, and to address the bigger questions. For example, did the atmosphere in the lab create the pressure to cut corners? Or did the intensity of the tenure chase contribute? One way to address such questions might be through internal departmental discussions, in which everyone is free to admit mistakes, and discuss how to fix the problems instead of apportioning the blame.

Indeed, the authors suggest 6 strategies for championing research integrity at an institutonal level:

Adopt zero tolerance: “Social responsibility to the academic community and to the public who fund the research will be strengthened when it is apparent that an institution has a real commitment to integrity.”

Protect whistleblowers: “more than two-thirds of whistleblowers, in a Research Triangle Institute study, experienced at least one negative outcome as a direct result of their actions. Plus, 43% reported that institutions encouraged them to drop the allegation.”

Clarify how to report: Establish “a reporting system that clearly identifies the individuals to whom allegations should be brought, and establishing clear policies, procedures and guidelines related to misconduct and responsible conduct.”

Train the mentors: “Mentors specifically need to become more aware of their roles in establishing and maintaining research rules and minimizing opportunities to commit research misconduct. An institutional investment in building better mentors is an important vehicle to promoting research integrity.”

Use alternative mechanisms: “Auditing research records would be one such means. Mechanisms of review are needed to reduce deficient record keeping, improper protection of human or animal subjects or the utilization of questionable research behaviour.”

Model ethical behaviour: “Institutions successfully stop cheating, for example, when they have leaders who communicate what is acceptable behaviour, encourage faculty members and staff to follow the policies, develop fair and appropriate procedures for handling misconduct cases, focus on ways to develop and promote ethical behaviour, and provide clear deterrents that are communicated.”

In this same issue, Nature also cites the decision of the Ottawa Health Research Institute to suspend Kristin Roovers, whose case ORI closed and reported on last year.

Of course, the lively discussion of the Hellinga retraction is just that - a discussion of the retraction and the science involved, though I suspect we will eventually be discussing more formal issues of misconduct.

Although the NYT has not printed a correction, clarification, editor’s note, retraction, or any other indication that the veracity of Mr. Finder’s reporting has been called into question, they did offer, rightly, the opportunity for VCU to respond in the form of a letter to the editor from President Trani. This letter, although more thoughtfully articulated than his prior memo, continues to downplay the real problems of any university engaging in research service agreements and the even greater ethical problems with this particular research service agreement, setting aside the sponsor for now.

As part of my commentary on this, I disclose that I have been sent a copy of the agreement obtained by a third party through a Va FOIA request (details below) and will quote verbatim a few critical points omitted by Trani in his letter and overall response to this story.

For example, secrecy (from the agreement itself):

”19. Neither party shall, without the prior written approval of the other party, (i) advertise or otherwise publicize in a written manner the existence or terms of this AGREEMENT or any TASK ORDER or any other aspect of the relationship between SPONSOR and VCU … If at any time a third party, including without limitation any news organization, contacts VCU concerning SPONSOR, VCU shall make no comment and shall notify promptly SPONSOR of the third party.”

Read the rest of this entry »

Update: I’ve discussed President Trani’s letter to the NYT above. And perhaps this NYT article explains how this all could have happened.

Update: Outstanding Blog Entry by an VCU Adjunct Professor summarizing the many (& snowballing) unethical activities going on in Richmond (and a great follow-up commentary here as well).

The NYT today reports on an astoundingly unethical and outrageous research agreement between VCU (Virginia Commonwealth University) and Philip Morris USA. The need to deny existence of this agreement if asked by the press … the ability of Philip Morris to remove everything but prepositions from journal manuscripts and any other form of dissemination … the pre-assignment to the company of intellectual property generated. Jiminy Crickets.
Read the rest of this entry »

Notice is hereby given that the Office of Research Integrity and the Acting Assistant Secretary for Health have taken final action in the following case:

Lois Bartsch, PhD, former postdoctoral research trainee, Department of Genetics, Cell Biology, and Anatomy, University of Nebraska Medical Center, engaged in scientific misconduct in research supported by grants P30 CA36727, R01 CA77876, and P20 RR016469. Specifically, PHS found that Dr. Bartsch:

Falsified DNA sequence files by deleting a nucleotide and changing nucleotide designations and reported the altered file as the ACI rat p16Cdkn2a sequence with a CpG dinucleotide polymorphism in the upstream region to GenBank, in grant application CA118151, and in the poster presented to Cold Spring Harbor Laboratory;

Fabricated the claim in grant application CA118151 that GenBank entries for the human p16Cdkn2a gene had a CpG polymorphism near the transcription start site;

Falsified the differential methylation of CpG dinucleotides near the transcription start site of p16Cdkn2a DNA and reported that tumor tissue was more methylated than normal tissue in ACI rats treated with estrogen and that the ACI allele was more methylated than the BN allele in tumor tissue from (BN x ACI)F1 animals treated with estrogen in grant application CA118151.

No, I’m not providing such … this was the conference I attended today. Needless to say, one full day wasn’t near enough, but when you don’t have industry sponsorship … The endowed lecture, this year entitled “How Financial Conflicts of Interest Endanger Our Profession”, was given by Jerome Kassirer, MD (Prof of Med, Tufts SOM; Visiting Prof, Stanford SOM; Editor-in-Chief Emeritus, NEJM). An equally fascinating keynote address was delivered by George Lowenstein, PhD (Herbert A. Simon Prof of Economics and Psychology, Dept of Social & Decision Sciences, Carnegie Mellon) on “A Psychological Perspective on Conflicts of Interest in Medicine.”

There was a consensus on the failure of disclosure as a means to remedy COI, despite its perception as the “great disinfectant.” Dr. Lowenstein presented a series of, yes, elegant studies demonstrating motivational information processing (brain processes information differently depending on whether processor wants to embrace or discard the information presented), that physicians are willingly misled even when they know the information being presented is biased, and that disclosure of a COI perversely corrupts the behavior, decisions, & actions of both the discloser and the disclosee. Dr. Kassirer added that disclosure alone requires an assessment of motives (difficult if not impossible) and gives the discloser license to proceed “unconflicted” in an “anything goes” mode.
Read the rest of this entry »

Two items in Nature journals consider the benefits of a consortium approach to peer review of journal manuscripts (in existence) and grant applications (modest proposal). Nature Neuroscience announced today it is joining the the Neuroscience Peer Review Consortium, which “reduces the overall reviewing workload of the community by allowing authors to continue the initial review process when their paper moves from one consortium journal to another, once the paper has been rejected or withdrawn from the first journal.” The Nature Neuroscience editorial describes the process, voluntariness, and flexibility and notes that the NPRC system will be evaluated at the end of the year … and on an ongoing basis at the journal’s blog, Action Potential.

Separately and quite distinctly, in a letter to Nature, Dr. Noam Harel of Yale makes a modest proposal: a centralized grant proposal repository into which applications could be deposited at the PI’s leisure and that sponsors could search for interesting science to review and possibly fund (no doubt with some encouragement by depositing PIs). The research proposals would only be made available to sponsor agencies, and multiple sponsors interested in the same work could collaborate on a shared funding agreement. As a thought exercise, interesting. As something to actually implement …

And finally, while we’re pondering peer review, Gregory Cuppan, a managing principal at McCulley/Cuppan (which specializes in document development), contributes to the commentary on a prior thread discussing the Publishing Research Consortium survey data. Specifically, he notes that “most people have little or no formal training in the task of review” and would “be interested to know how many readers of this blog have actual formal training in the task of review (here I make a strong distinction from training for the task of editing).” He refers to a 1961 study by the Educational Testing Service in which 53 distinguished reviewers read 300 college student papers but only had a median correlation among reviewer scores of 0.31.

In a separate note to me, he also suggested we look at an article by Mayo et al. in the Journal of Clinical Epidemiology suggesting traditional grant review processes and funding decisions suffer from a high degree of variability due to too few reviewers being involved. The report presents empirical data from intramural review of pilot project applications at McGill University Health Center Research Institute; applications were both ranked and scored (1-5 scale), with poor agreement between the two (kappa value of 0.36, with 95% CI 0.02-0.70). The top-ranked proposals would have failed to meet the “payline” with varying probability depending on who was assigned to provide a scored review. The examined process does not translate to current NIH study section practice, but it lends credence to the recommendation (see pp 4-5 & 38-41) that chartered study section members (not ad hoc reviewers) rank scored proposals at the end of each meeting. Per Mr. Cuppan’s suggestion that manuscript reviewers lack training, see pp 45-46 of the Enhancing Peer Review report for standardizing reviewer, chair, and administrator (officer) training.

Latest Discussion can be read/joined here.

I wanted to thank David for his many comments in the ongoing discussion of the Dwyer et al. retractions, including his pointing out that the 2007 Journal of Molecular Biology article from this group was also formally retracted and, most recently, two enlightening electronic letters to Science (particularly the one from Dr. Richard at SUNY Buffalo) on this case.

And yet another update via David in the main thread on this retraction:

“One very interesting section of Andrea’s [Gawrylewski, The Scientist] blog entry that has not appeared online before is that “Shortly after the original paper appeared in Science in 2004, Hellinga went to give a seminar in Berkeley to present his new findings. [Jack] Kirsch said he brought up the issue with the Km and asked to see Hellinga’s data but never received it.” Yet another lesson to learn from this whole fiasco.”

Zarin and Tse (from NLM) have published in Science a thoughtful commentary, Moving Toward Transparency of Clinical Trials, in which they discuss problems of reporting bias against negative trials and review Section 801 of the FDA Amendments Act (”FDAAA 801″), which was enacted on Sept 27, 2007 and “expands the scope of required registrations at ClinicalTrials.gov and provides for the first federally funded trial results database. It mandates registration of a set of controlled clinical investigations, other than phase I trials, of drugs, biologics, and devices subject to regulation by the FDA. The law applies to research for any condition regardless of sponsor type (e.g., industry, government, or academic).”

This is a massive undertaking of great import, and they conclude that “FDAAA 801 should transform the degree of public access to critical clinical trial information from publicly and privately funded clinical research” but also discuss several challenges to its implementation related to potential loopholes and issues of IP, validation, and interpretation by the public. For example, the “NIH and FDA are directed by the law to conduct a pilot quality-control project to determine the optimal method of verification” of the completeness and accuracy of submitted data. Whoa. And double whoa when it comes to balancing the need to protect IP while also protecting and promoting public health (particularly those who volunteer to participate in clinical research). Stay tuned.

Nature Nanotechnology continues the discussion of journal peer review with a summary of and commentary on the recent Publishing Research Consortium report. Letters to Nature continue the discussion of double-blind peer review (pro, compromise, unworkable, need better data), as does the accompanying online discussion forum. [This same issue of Nature also includes some letters addressing duplication and plagiarism, and you all are encouraged to join the ongoing online debate on this matter.] Letters to Science bring suggestions for rewarding and rating journal reviewers (with, unfortunately, no linked discussion as is available at Nature).

And on the heels of intense discussion about the Dwyer et al. retraction, Science now publishes a “Letter of Concern” (a pre-retraction of sorts - “Science is publishing this Editorial Expression of Concern to alert our readers to the fact that serious questions have been raised about the validity of the findings in the Won et al. paper.”) the same week that Nature formally retracted one of its own reports when the authors “lost confidence in the reported conclusions … [and in turn] regret any adverse consequences that may have resulted from the paper’s publication.”.

Update: Science provides an overview of this case as well.

As reported in Chemical & Engineering News, “A CHEMIST IN INDIA has been found guilty of plagiarizing and/or falsifying more than 70 research papers published in a wide variety of Western scientific journals [n=25] between 2004 and 2007.”

The unprecedented fraud by Pattium Chiranjeevi of Sri Venkateswara University was discovered by Purnendu Dasgupta, the US editor of Analytica Chimica Acta, when a reviewer noted that a Chiranjeevi submission was similar to a published paper from a Japanese group. “In fact, Dasgupta says, but for the change in the name of the chemical being measured, the papers were identical.” Elsevier has retracted the 13 Chiranjeevi’s articles published in its journals.

In addition to trying to blame his students for the plagiarism, Chiranjeevi “was charging students a fee to award them degrees” - not sure if this means in addition to the University tuition or what, but maybe it was for the bonus lessons in scientific misconduct: “some of them [his students] were aware of and participated in the fraud he perpetrated.”

Dasgupta reported using eBlast, which is used to develop/maintain the Deja Vu database, to find reviewers but, interestingly, seems only after the fact to have considered it as a means to flag potential plagiarism.