Science has a nice recap of how the current continuing resolution (through March 6, 2009) and ongoing budget dance affects federal agencies that sponsor research. The biggest loser: the NIH. Read the rest of this entry »
Archive for September, 2008
Sciencedebate 2008
Update: Nature looks at likely Obama administration budget priorities – and realities.
Update: Thoughtful & well-written Nature editorial endorsing Obama.
In anticipation of the start of the Presidential debates, excerpts from Sciencedebate 2008: Read the rest of this entry »
Thus Spake Zerhouni
So, I wasn’t going to bother to post about this, given the ubiquitous coverage in the media and the blogosphere, but I might as well point you to pieces in the NYT and Science and The Scientist, think about what’s next, and settle in for the rest of the ride.
Given that the Great Zerhouni, like all political appointees, would have been submitting his resignation come January 2009, this is a bit odd he would bail a few months shy, especially since Kessler as FDA director set the precedent of straddling administrations (& parties in power; h/t BB). If the GZ had wanted to stay, it wasn’t out of the question. And with the CTSA Consortium just about to fill up and the T-R01s pouring in next January, I’m a bit surprised. Plus, if he truly wanted as minimal disruption as possible, he would have waited for the next appointed director to be approved by the Senate before stepping down. Someone will have the thankless job of serving as interim director for 6-8 months (or a couple of years, as warned by Nature).
One could look at the spate of conflict of interest issues and pressure by Grassley as inspiration to spend more quality time with his family, but I suspect it more likely that the GZ wants to handle his departure on his terms before the election fireworks (perhaps even participate in a nonpartisan fashion?) … and before the budget impasse gets underway. I wouldn’t blame him for taking a pass on that at all.
I am among those who are grateful for what the Great Zerhouni has done for the NIH, though I think opportunities for even greater impact on US biomedical research remain for the next Director. Such as? I’m sure the pundits will be outing folks soon. My only grapevine whisper has been Steve Hyman, the Provost at Harvard. I think the AAMC is right on in hoping the GZ’s successor “will continue to be a strong advocate for a sustainable, predictable, and increasing investment in NIH research.” I like the “sustainable, predictable” part as much as the “increasing”.
In the meantime, we have the Great Zerhouni’s message to the NIH folks themselves:
From: Zerhouni, Elias (NIH) [E]
Sent: Wednesday, September 24, 2008
To: NIH-STAFF-URGENT@LIST.NIH.GOV
Subject: Announcement and Message of AppreciationDear Friends and Colleagues:
For over six years, I have had the privilege of leading one of the greatest institutions in the world, the National Institutes of Health, the “Nation’s Medical Research Agency.” I have decided, however, that it is time for me to turn my attention to new opportunities, including several writing projects.
NIH is one of the true “wonders of the world,” and its strength comes from you, our scientists, administrators, staff, contractors, and trainees. As NIH Director, I have had the unparalleled privilege of working with the most extraordinary staff in the world, each one of you dedicated to a great, single mission: improving the nation’s health. Every one of you has played an integral part in our ability as an agency to build upon an outstanding record of achievement while creating new inroads that will continue to pay off for years to come. I want to express my sincere thanks for your support and your spirit of commitment, cooperation, and resourcefulness during my term as NIH director. It has made all the difference during my time here
I admire your unparalleled dedication to advancing innovative research, fostering scientific collaboration, and enhancing basic and clinical research for the benefit of people everywhere. I feel a special debt to the people I have worked with most closely-the Institute and Center Directors and my staff in the Office of the Director.
To everyone at NIH, thank you for your support. Together, we are experiencing a true revolution in the biomedical sciences, one that continues to have broad and profound implications for human health. I am extremely fortunate to have led the agency that has been at the center of this revolution, and an agency that met every challenge put to it as a result of that revolution.
I will be leaving NIH by the end of October, and Secretary Leavitt and I are working together to ensure an orderly transition in the weeks ahead. We will keep you informed of plans as they develop.
Elias A. Zerhouni, M.D.
Drugs, But No Smokes
Given that the Senate is a tad preoccupied with other matters just now, the Family Smoking Prevention and Tobacco Control Act may not be brought up for a vote this fall. In the meantime, though, San Francisco has passed a ban on the sale of tobacco in pharmacies starting in October 2008.
Writing in JAMA, Mitchell Katz notes that “82% of pharmacists and 72% of adult consumers surveyed in California believe that pharmacies should not sell tobacco.”
This is nothing new: in Canada, 7 provinces (Ontario, Quebec, New Brunswick, Nova Scotia, Nunavut, Newfoundland-Labrador, and Prince Edward Island) have banned the sale of tobacco in pharmacies, beginning with Ontario in 1993. Of note, the ban had no negative economic impact on pharmacies in these provinces, and in fact they flourished. Indeed, Katz indicates that “97% of California consumers reported that they would continue to patronize their pharmacy as often or even more often if the pharmacy stopped selling tobacco products.”
Regarding the ubiquitous slippery slope argument, Katz notes:
Pharmacies sell other products that may be associated with adverse health effects. For instance, what about the sale of alcohol in pharmacies,given that there are 85 000 alcohol-related deaths per year in the United States?1 What about the sale of candy bars in pharmacies, given the epidemic of obesity? The difference between alcohol or food with high-fat and sugar content and tobacco is that there is no safe level of tobacco as there is with these other substances. Alcohol in moderation can be health promoting.
He goes on to add,
With a ban in place, pharmacy staff can offer the right response when customers ask for a pack of cigarettes:”I’m sorry this is a health-promoting business; we don’t sell tobacco. May I offer you advice on how to quit?”
If only academic medical centers would be so enlightened with regard to banning tobacco industry sponsorship of research and of student coursework.
Addressing Underpublication of Clinical Trial Results
In the September issue of The Oncologist, Scott Ramsey and John Scoggins (both from Fred Hutchinson) report that fewer than one in five cancer clinical trials registered with clinicaltrials.gov have been published in the peer-reviewed literature (17.9%). Categories of studies with the lowest rates of publication included industry-sponsored (5.9%), non-randomized (4.4%), and terminated (3.4%) trials.
In examining the 357 trials with published results, the authors could judge whether the results were positive or negative for 341. The majority (64.5%) reported positive results, with Phase I trials most likely to report positive results (89.9%), followed by Phase IV (83.3%), Phase III (63.2%), and Phase II (53.6%). NIH-sponsored trials were most likely to result in positive results (78.8%).
Previously, Richard Johnson and Kay Dickersin (both from Johns Hopkins) also raised the issue of publication bias against negative clinical trials in Nature Clinical Practice Neurology.
Fortunately, the FDA Amendments Act of 2007 has set in motion the expansion of clinicaltrials.gov to include compulsory reporting of basic results. You can check out progress made on the basic results data entry test system, which is designed to capture the following information:
‘‘(i) DEMOGRAPHIC AND BASELINE CHARACTERISTICS OF PATIENT SAMPLE.—A table of the demographic and baseline data collected overall and for each armof the clinical trial to describe the patients who participated in the clinical trial, including the number of patients who dropped out of the clinical trial and the number of patients excluded from the analysis, if any.‘
(ii) PRIMARY AND SECONDARY OUTCOMES.—The primary and secondary outcome measures as submitted under paragraph (2)(A)(ii)(I)(ll), and a table of values for each of the primary and secondary outcome measures for each arm of the clinical trial, including the results of scientifically appropriate tests of the statistical significance of such outcome measures.
(iii) POINT OF CONTACT.—A point of contact for scientific information about the clinicaltrial results.
(iv) CERTAIN AGREEMENTS.—Whether there exists an agreement (other than an agreement solely to comply with applicable provisions of law protecting the privacy of participants) between the sponsor or its agent and the principal investigator (unless the sponsor is an employer of the principal investigator) that restricts in any manner the ability of the principal investigator, after the completion date of the trial, to discuss the results of the trial at a scientific meeting or any other public or private forum, or to publish in a scientific or academic journal information concerning the results of the trial.”
Indeed, in an accompanying commentary, James Doroshow notes that NCI is developing in parallel a complementary clinical trials database to catalogue administrative and outcome data for all studies performed at NCI-supported institutions (perhaps drawing from or building on the NCI’s excellent existing Cancer Research Portfolio database). Unlike the clinicaltrials.gov basic results reporting, the NCI database will include interim reports on accrual and outcomes. Thus, the oncology community will, within the next few years, have rapid access to safety and efficacy data from cancer clinical trials.
Further, the editors of The Oncologist, Gregory Curt and Bruce Chabner, indicate that they are considering whether to “undertake the publication of a peer-reviewed, searchable venue for these trials” in reference to “well-executed trials that fail to meet positive endpoints: ‘negative’ in a sense, but valuable nonetheless.” The editors invite readers to indicate their level of enthusiasm and support for such a venture.
The title of Doroshow’s commentary captures the urgency for action on this front, not only in the oncology community but among all clinical disciplines: Publishing Cancer Clinical Trial Results: A Scientific and Ethical Imperative.
Implementing Enhanced Peer Review
Update: NIAID has a nice table comparing the old and new peer review processes.
Today the NIH provided a timeline on when specific components of peer review reform will be formally implemented. Those of most interest to applicants include:
2009
- 1-7 scoring scheme & structured summary statement will commence in May.
- All triaged/streamlined applications will receive a score (preliminary, not discussed).
- All “Early Stage Investigator” applications will be clustered for review.
- Only one amended application allowed (i.e., only A0 & A1 submissions – no A2)
2010
- R01 applications shortened to 12 pages & realigned to match review criteria (ie, impact, investigator/s, innovation-originality, project plan-feasibility, environment) will commence in January.
- Shorten applications to other funding mechanisms based on 12-p R01 scale.
Under Consideration (no timeline)
- Separate percentiling of new and resubmitted applications.
- Cluster clinical research applications for review.
Trends in Disease Focus & Translational Timelines
A couple of recent reports may be of interest to the translational research crowd. Nature Reviews Drug Discovery reported Trends in Disease Focus of Drug Development using data from Phase II-IV studies registered with clinicaltrials.gov from Sept 2005 through Sept 2007 (as a reminder, the ICMJE requires registration of all studies to be reported in journals abiding by their policies).
Six therapeutic areas accounted for two-thirds of all protocols: oncology, central nervous system disorders, cardiology, infectious diseases, endocrinology, and respiratory diseases. Of these six, respiratory diseases, endocrinology and oncology showed growth in the number of trials registered during this period. Among all disease categories, relative growth was greatest for rheumatology (157.6%).
On this side of the pond, Science published a Policy Forum article on the Life Cycle of Translational Research for Medical Interventions confirming the time and effort needed for the drug discovery efforts noted above to translate into accepted effective interventions. Indeed, the authors start off with the stark observation that:
Of 101 very promising claims of new discoveries with clear clinical potential that were made in major basic science journals between 1979 and 1983, only five resulted in interventions with licensed clinical use by 2003 and only one had extensive clinical use.
Matched by a thoughtful conclusion:
Successful translation is demanding and takes a lot of effort and time even under the best circumstances; making unrealistic promises for quick discoveries and cures may damage the credibility of science in the eyes of the public.
The authors sought out the first highly cited clinical study published showing effectiveness of a specific intervention (i.e., received over 1000 citations in the literature in 1990-2004 on the basis of the Web of Science). Of the 49 articles in this category, 32 reported the intervention investigated as effective and were selected for further examination (if more than one highly cited article covered the same intervention, only the first report was selected). By the end of 2006, the effectiveness of 19 interventions had been replicated by subsequent studies (14) or had remained unchallenged (5), whereas the other 13 had been either contradicted (5) or found to have had initially stronger effects (8) when larger or better controlled subsequent studies were performed.
To estimate the “translation lag” for each intervention to reach this milestone (publication cited at least 1000 times), the authors went back to identify the year of the earliest journal publication on preparation, isolation, or synthesis appeared or the earliest patent was awarded (whichever occurred first).
Using these milestones, “the median translation lag was 24 years (interquartile range, 14 to 44 years) between first description and earliest highly cited article.” Repeating the analysis using the time of discovery as the first publication or awarded patent of any agent in the wider intervention class showed even greater lag: median of 27 years (interquartile range, 21 to 50). Not surprisingly, among 18 nonrefuted interventions, the median translation lag was 16.5 years (range 4 to 50 years) or 22 years (range 6 to 50 years) when considering the wider class.
The authors note that
The fastest successful translation occurred for indinavir (as part of triple antiretroviral therapy) and abciximab, both of which took only 4 years from their patenting to the publication of a highly cited randomized trial. Both of these fast successes involved multidisciplinary work spanning molecular to clinical research on protease inhibitors and integrins, respectively.
On the other hand, they note that rather than continue to study highly cited claims that were eventually refuted (e.g. flavonoids, vitamin E, estrogen), time, money, and effort would be better invested in novel agents and technologies for treating for common diseases since “it is unlikely that genuine major benefits from interventions already known for a long time have gone unnoticed.”
Of course, a recent article in Circulation notes that, in addition to being more likely to report positive results, industry-sponsored cardiovascular clinical trials are more likely to be cited in the literature than studies performed by not-for-profit organizations (e.g., the NIH). The HeartWire summary notes that
The for-profit organizations have better media-outreach programs in place, ensuring wider press coverage that ultimately increases the citation rate. These companies also invest more resources to increase the number of secondary publications for a trial, leading to more exposure of the initial study.
It would be interesting to compare the translational lag of industry versus publicly funded research.
Journal Editorial Process and Publishing Standards
An editorial in Nature Structural & Molecular Biology examines the editorial process in an effort to shed some light on why manuscript reviews take as long as they sometimes (often) do. In a perfect world, the journal would make an editorial decision about 3 weeks after submission. In a perfect world.
Of course, NSMB notes that their initial effort to be speedy often incurs the ire of submitting authors: once an editor is assigned to a manuscript (often on the day of submission), they decide within 1-2 days whether to accept the paper for review. If reviewers are flagged as incompetent, I’m sure these authors rejected out of hand have some choice descriptors for the editors involved. And, indeed, this would involve most investigators sending manuscripts to NSMB, where approximately 80% of submissions are not reviewed. The rationale for this selectivity is to avoid reviewer fatigue.
Among manuscripts accepted for review, the problem becomes one of finding 2-3 willing and able reviewers, a task that can take a week at NSMB (which seems relatively quick to me). Although the editors request reviewer comments within 14 days of assignment, this step can instead drag out over several weeks. Resnick et al. reported that 9.6% of their survey respondents felt journal reviewers intentionally delayed review of the assigned manuscript so as to get to press first, but the NSMB editors feel it more likely that late reviews can simply be attributed to the very busy lives of their reviewers. NSMB also notes that “when authors alert us about specific competition, we can expedite the review process to make a decision in less than 10 days after submission.”
Given the complexity of the science in manuscripts accepted for review, often journal editors must rely on reviewers with special expertise and have little recourse for their tardiness. Using only those reviewers who submit their comments promptly would result in a very small pool – which would evaporate further as these folks, burdened with more and more review assignments, realized that no good deed goes unpunished.
Thus, in an era of instant messaging and twittering, communication of science continues at a more deliberate pace to ensure the message is suitable for the target journal audience. Indeed, Bruce Alberts reminds us in Science that “publication of a scientific article is less a way for scientists to earn recognition and advance their careers than it is an engine for scientific progress” and asks that “reviewers and editors of scientific manuscripts … constantly ask themselves whether the reader has been provided with everything needed to both understand and reproduce the results.”
He goes on to note that “journals themselves can certainly set a higher bar for the clarity of presentation in the manuscripts that we publish.” I suspect the submission of more clearly and thoughtfully written manuscripts would make reviewers happier – and speedier – as well.
Alberts has a kindred spirit in Linda Cooper of McGill University, who in her letter to Nature raises the concern that “papers published today are less successful in meeting their objectives [communicating scientific discoveries] than in the past,” noting that “most published papers still compress too much information into uncomfortably short articles, leading to convoluted sentences, specialized terminology and a proliferation of abbreviations.”
Alberts in Science similarly acknowledges that “Some abstracts, full of three-letter abbreviations and jargon, are incomprehensible to me even in my own field of cell biology.”
Back across the pond in Nature, Cooper laments that such trends spill over into the electronic publishing where “online manuscripts are often bound by the same space constraints as print manuscripts.” Of course, the supplemental electronic material must also be peer-reviewed so cannot be limitless (editors must remain considerate of their reviewers), but perhaps use of JoVE and the like could lessen reviewer burden in terms of needing to plow through dense methods sections and improve the clarity of communication of scientific discovery via embedded videos.
