Archive for Biomedical Writing/Editing

No Gender Bias in Peer Review

May 28, 2009 at 5:24 pm · Filed under Biomedical Research Ethics, Biomedical Writing/Editing, Research News

A piece in this week’s issue of Nature by Herbert Marsh (professor of education at the University of Oxford, UK) and Luz Bornmann (PhD student at the ETH University in Zurich, Switzerland) asks (and answers) the question, Do women have less success in peer review? The Naturejobs item is so brief that I include it here, hopefully without raising the copyright ire of Nature Publishing Group.

Peer review assesses what is of value in science, yet it has been widely criticized for biases. One such perceived bias is gender. But evidence for such a bias has been contradictory. A 2007 meta-analysis (L. Bornmann et al. J. Informet. 1, 226–238; 2007; see also Nature 445, 566; 2007) concluded that women are at a disadvantage in peer review. As this study incorporated all known research on this issue, it seemed a definitive answer.

However, a study published last year (H. W. Marsh et al. Am. Psychol. 63, 160–168; 2008) presented conflicting results. It was the most comprehensive primary-research study, based on data from the Australian Research Council (10,023 reviews by 6,233 external assessors of 2,331 proposals from all disciplines). The study found that the gender of the applicant had no effect on the outcomes of peer review, irrespective of the discipline, the gender and nationality of the reviewers, and whether reviewers were selected by a funding panel or chosen by the applicants.

Why should these two studies have conflicting results? To investigate, both research teams worked together to reanalyse the data and extend the original meta-analysis. We applied new, stronger statistical approaches to 66 sets of results representing 353,725 proposals from 8 countries. In this extended study, which will be published in Review of Educational Research, we found no effect of the applicant’s gender on the peer review of their grant proposals. This lack of effect held across country, year of publication of the studies included in the meta-analysis, and disciplines ranging from physical sciences to the humanities.

The study did, however, reveal very small — but statistically significant — gender differences in favour of men for the 26 sets of results that were for fellowship applications. However, these fellowship results varied greatly between the individual studies within the analysis, indicating that they are not generalizable. We suggest that the differences might have arisen because fellowship applicants tend not to have established a solid track record in their research. In the absence of sound evidence on which to base their judgements, peer reviewers might therefore have been influenced by irrelevant characteristics such as gender.

At least for grant applications, all of the co-authors from each of the research teams agree that the weight of evidence suggests that the applicant’s gender has no effect on the outcome of peer review, and that these findings are robust and broadly generalizable.

Of possible interest is the fact that male authors are the ones examining these issues … I wonder if on this side of the pond, Molly Carnes and her cronies might opt to take a look as well.

Comments (3)

What Journal Editors Really Want

May 18, 2009 at 4:21 pm · Filed under Biomedical Writing/Editing, Research News

The obvious, apparently, as reported by David Shieh in The Chronicle of Higher Education, particularly at AMA and Nature journals.

SINK OR SWIM: Young children who take swimming lessons are less likely to die of drowning than those who don’t take lessons, found a study published in the March issue of the Archives of Pediatrics & Adolescent Medicine. Researchers, who interviewed the families of 88 drowning victims in six states, conceded that their study did not explain why the correlation exists, but noted that “it seems reasonable to assume that at least part of the protective effect is through increased swimming skills.”

BOREDOM EVERLASTING: Bored after seven years of marriage? Little will change after nine more, says a study published last month in Psychological Science. In a study of 123 married couples, researchers reported that boredom in the seventh year of marriage “strongly and significantly predicted less satisfaction” in the 16th year. Luckily, researchers also found a way to save marriages from being eroded by boredom: adding “excitement” to the relationship.

SWEDE TOOTH: Those who wake up in the middle of the night to head for the refrigerator are more likely to be obese, reports a study in this month’s issue of the journal Obesity. Researchers, who teamed up with the Swedish Twin Registry in Stockholm to survey more than 20,000 Swedish twins for the study, found that “night eating” was also correlated with binge eating, disrupted sleep patterns, and insomnia.

IS THAT A BENTLEY IN YOUR POCKET? Men who drive luxury cars are found to be more attractive than those who drive subcompacts, says a study published online in March in the British Journal of Psychology. Study participants were shown pictures of a model of the opposite sex in two different cars: a silver Bentley Continental and a red Ford Fiesta. While men found the model equally attractive in both settings, women rated the model as “significantly more attractive” when sitting in the Bentley. Noted the study: “It would appear that despite a noticeable increase in female ownership of prestige/luxury cars over recent years, males, unlike females, remain oblivious to such cues.”

Of course, this last journal also just published Men’s preferences for women’s profile waist-to-hip ratio, breast size, and ethnic group in Britain and South Africa, which reported “a strong positive correlation between ratings of attractiveness and health.” No, I do not know whom you should contact about volunteering to participate in this sort of research.

Comments (2)

Winner’s Curse: Why Sensational but Inaccurate Findings Get Published

November 23, 2008 at 2:58 am · Filed under Biomedical Research Ethics, Biomedical Writing/Editing, Research News

This week’s edition of On the Media included a piece entitled Bad Study Habits that discussed a recent report in The Economist on why much published research is wrong:

IN ECONOMIC theory the winner’s curse refers to the idea that someone who places the winning bid in an auction may have paid too much. Consider, for example, bids to develop an oil field. Most of the offers are likely to cluster around the true value of the resource, so the highest bidder probably paid too much.

The same thing may be happening in scientific publishing, according to a new analysis. With so many scientific papers chasing so few pages in the most prestigious journals, the winners could be the ones most likely to oversell themselves—to trumpet dramatic or important results that later turn out to be false. This would produce a distorted picture of scientific knowledge, with less dramatic (but more accurate) results either relegated to obscure journals or left unpublished.

Read the rest of this entry »

Comments (4)

Newsless Papers

October 29, 2008 at 2:01 am · Filed under Biomedical Writing/Editing, Research News

What if you opened your New York Times and discovered it was populated almost entirely with AP, Reuters, UPI, and other wire stories? What if the LA Times, the Washington Post, and the Chicago Tribune all looked about the same? Good-bye Science Tuesday. Hello poor new world of newspaper publishing. Or not even that perhaps. Read the rest of this entry »

Leave a Comment

Addressing Underpublication of Clinical Trial Results

September 22, 2008 at 6:33 pm · Filed under Biomedical Research Ethics, Biomedical Writing/Editing, Research Resource

In the September issue of The Oncologist, Scott Ramsey and John Scoggins (both from Fred Hutchinson) report that fewer than one in five cancer clinical trials registered with clinicaltrials.gov have been published in the peer-reviewed literature (17.9%). Categories of studies with the lowest rates of publication included industry-sponsored (5.9%), non-randomized (4.4%), and terminated (3.4%) trials.

In examining the 357 trials with published results, the authors could judge whether the results were positive or negative for 341. The majority (64.5%) reported positive results, with Phase I trials most likely to report positive results (89.9%), followed by Phase IV (83.3%), Phase III (63.2%), and Phase II (53.6%). NIH-sponsored trials were most likely to result in positive results (78.8%).

Previously, Richard Johnson and Kay Dickersin (both from Johns Hopkins) also raised the issue of publication bias against negative clinical trials in Nature Clinical Practice Neurology.

Fortunately, the FDA Amendments Act of 2007 has set in motion the expansion of clinicaltrials.gov to include compulsory reporting of basic results. You can check out progress made on the basic results data entry test system, which is designed to capture the following information:

‘‘(i) DEMOGRAPHIC AND BASELINE CHARACTERISTICS OF PATIENT SAMPLE.—A table of the demographic and baseline data collected overall and for each armof the clinical trial to describe the patients who participated in the clinical trial, including the number of patients who dropped out of the clinical trial and the number of patients excluded from the analysis, if any.‘

(ii) PRIMARY AND SECONDARY OUTCOMES.—The primary and secondary outcome measures as submitted under paragraph (2)(A)(ii)(I)(ll), and a table of values for each of the primary and secondary outcome measures for each arm of the clinical trial, including the results of scientifically appropriate tests of the statistical significance of such outcome measures.

(iii) POINT OF CONTACT.—A point of contact for scientific information about the clinicaltrial results.

(iv) CERTAIN AGREEMENTS.—Whether there exists an agreement (other than an agreement solely to comply with applicable provisions of law protecting the privacy of participants) between the sponsor or its agent and the principal investigator (unless the sponsor is an employer of the principal investigator) that restricts in any manner the ability of the principal investigator, after the completion date of the trial, to discuss the results of the trial at a scientific meeting or any other public or private forum, or to publish in a scientific or academic journal information concerning the results of the trial.”

Indeed, in an accompanying commentary, James Doroshow notes that NCI is developing in parallel a complementary clinical trials database to catalogue administrative and outcome data for all studies performed at NCI-supported institutions (perhaps drawing from or building on the NCI’s excellent existing Cancer Research Portfolio database). Unlike the clinicaltrials.gov basic results reporting, the NCI database will include interim reports on accrual and outcomes. Thus, the oncology community will, within the next few years, have rapid access to safety and efficacy data from cancer clinical trials.

Further, the editors of The Oncologist, Gregory Curt and Bruce Chabner, indicate that they are considering whether to “undertake the publication of a peer-reviewed, searchable venue for these trials” in reference to “well-executed trials that fail to meet positive endpoints: ‘negative’ in a sense, but valuable nonetheless.” The editors invite readers to indicate their level of enthusiasm and support for such a venture.

The title of Doroshow’s commentary captures the urgency for action on this front, not only in the oncology community but among all clinical disciplines: Publishing Cancer Clinical Trial Results: A Scientific and Ethical Imperative.

Comments (5)

Journal Editorial Process and Publishing Standards

September 10, 2008 at 12:48 am · Filed under Biomedical Research Ethics, Biomedical Writing/Editing

An editorial in Nature Structural & Molecular Biology examines the editorial process in an effort to shed some light on why manuscript reviews take as long as they sometimes (often) do. In a perfect world, the journal would make an editorial decision about 3 weeks after submission. In a perfect world.

Of course, NSMB notes that their initial effort to be speedy often incurs the ire of submitting authors: once an editor is assigned to a manuscript (often on the day of submission), they decide within 1-2 days whether to accept the paper for review. If reviewers are flagged as incompetent, I’m sure these authors rejected out of hand have some choice descriptors for the editors involved. And, indeed, this would involve most investigators sending manuscripts to NSMB, where approximately 80% of submissions are not reviewed. The rationale for this selectivity is to avoid reviewer fatigue.

Among manuscripts accepted for review, the problem becomes one of finding 2-3 willing and able reviewers, a task that can take a week at NSMB (which seems relatively quick to me). Although the editors request reviewer comments within 14 days of assignment, this step can instead drag out over several weeks. Resnick et al. reported that 9.6% of their survey respondents felt journal reviewers intentionally delayed review of the assigned manuscript so as to get to press first, but the NSMB editors feel it more likely that late reviews can simply be attributed to the very busy lives of their reviewers. NSMB also notes that “when authors alert us about specific competition, we can expedite the review process to make a decision in less than 10 days after submission.”

Given the complexity of the science in manuscripts accepted for review, often journal editors must rely on reviewers with special expertise and have little recourse for their tardiness. Using only those reviewers who submit their comments promptly would result in a very small pool – which would evaporate further as these folks, burdened with more and more review assignments, realized that no good deed goes unpunished.

Thus, in an era of instant messaging and twittering, communication of science continues at a more deliberate pace to ensure the message is suitable for the target journal audience. Indeed, Bruce Alberts reminds us in Science that “publication of a scientific article is less a way for scientists to earn recognition and advance their careers than it is an engine for scientific progress” and asks that “reviewers and editors of scientific manuscripts … constantly ask themselves whether the reader has been provided with everything needed to both understand and reproduce the results.”

He goes on to note that “journals themselves can certainly set a higher bar for the clarity of presentation in the manuscripts that we publish.” I suspect the submission of more clearly and thoughtfully written manuscripts would make reviewers happier – and speedier – as well.

Alberts has a kindred spirit in Linda Cooper of McGill University, who in her letter to Nature raises the concern that “papers published today are less successful in meeting their objectives [communicating scientific discoveries] than in the past,” noting that “most published papers still compress too much information into uncomfortably short articles, leading to convoluted sentences, specialized terminology and a proliferation of abbreviations.”

Alberts in Science similarly acknowledges that “Some abstracts, full of three-letter abbreviations and jargon, are incomprehensible to me even in my own field of cell biology.”

Back across the pond in Nature, Cooper laments that such trends spill over into the electronic publishing where “online manuscripts are often bound by the same space constraints as print manuscripts.” Of course, the supplemental electronic material must also be peer-reviewed so cannot be limitless (editors must remain considerate of their reviewers), but perhaps use of JoVE and the like could lessen reviewer burden in terms of needing to plow through dense methods sections and improve the clarity of communication of scientific discovery via embedded videos.

Comments (2)

Journal Peer Reviewers: Incompetent & Biased

August 27, 2008 at 12:57 pm · Filed under Biomedical Research Ethics, Biomedical Writing/Editing

So reports an article by intramural researchers at NIEHS in this month’s issue of Science & Engineering Ethics entitled, “Perceptions of Ethical Problems with Scientific Journal Peer Review: An Exploratory Study.” The authors – David Resnik, Christina Gutierrez-Ford, and Shyamal Peddada – conducted an anonymous survey of participants at mandatory RCR training sessions at NIEHS (51% response rate). The average age of respondents was 42, with an average of 35 papers published.

Here were the questions and percentage of “yes” responses:

Have any of the following ever happened to you during the peer review process?

    A reviewer was incompetent – 61.8%
    A reviewer was biased – 50.5%
    A reviewer required you to include unnecessary references to his/her publication(s) – 22.7%
    Comments from reviewers included personal attacks – 17.7%
    A reviewer delayed the review so that he/she could publish an article on the same topic – 9.6%
    A reviewer breeched confidentiality – 6.8%
    A reviewer used your ideas, data, or methods without your permission – 4.5%

Read the rest of this entry »

Comments (11)

Author Contributions Revisited

August 26, 2008 at 12:11 am · Filed under Biomedical Research Ethics, Biomedical Writing/Editing

Five senior editors at the Croatian Medical Journal published a tidy randomized study of two methods for assessing authorship contribution in the Journal of General Internal Medicine. The study involved 865 authors of 181 manuscripts submitted to the journal (CMJ, that is) between January and July 2005 who were randomized to provide either binary (yes/no) or ordinal (0-4 scale) rating of their contributions. Each author on each paper had to complete and sign the form; the first or corresponding author could not serve as a proxy for the others. The qualitative values for the ordinal rating were “none” (0), “small” (1), “moderate” (2), “large” (3), and “full” (4).
Read the rest of this entry »

Comments (1)

Demise of the Lone Author

August 24, 2008 at 9:21 pm · Filed under Biomedical Writing/Editing

In reading a recent article on author contributions, I remembered an essay I had been meaning to set out for reflection in case it had been missed due to the timing of its publication (Dec 20, 2007). This would be Mott Greene’s essay in Nature entitled The Demise of the Lone Author.

Greene notes that “From the late 1600s until about 1920, the rule was one author per paper,” which ensured clarity regarding who received credit for which scientific advance … which in turn is important to research sponsors, promotion & tenure committees, search committees, potential trainees, and so forth. Greene’s concern with the proliferation of authors (reaching the 100s when sequences are published) is that few journals include notes about how each author contributed to the research and the article itself. With today’s cattle call authorships, it is hard to give serious weight to the contributions of those listed after the first author (or the first two when an asterisk emphasizes their equal contributions).

Greene turns to Lotka’s Law (Alfred Lotka, 1926) for mathematical relief, describing it as:

a rough ‘inverse-square law of scientific productivity’. For every 100 authors who each produce a scientific paper in a given period, there will be 25 authors who produce two, 11 who produce three, and one author who produces ten or more.

Thus, the more citations you have, the more prominent and reputable a scientist you are – and more worthy of support.

Then Greene, acknowledging modern trends, muddies the purity of mathematics with a law from the dismal science:

Goodhart’s law, from the economist Charles Goodhart: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Once citation counting became established as a means to determine prominence, players began to ‘game the system’ based on their knowledge of that standard, and the metric ceased to have a close relation to the outcome it was designed to measure. Such attempts led to the somewhat occult business of impact factors, impact journals, author rank within a paper, and other such countermeasures to re-establish the utility of citation counting.

An occult business indeed. However, Greene is concerned that even Lotka’s law combined with black magic to prevent “author gaming” cannot withstand the proliferation of papers with 100 or more authors. He envisions a sort of cinematic rolling of the credits to accommodate the casts of thousands and perhaps even institutionally imposed restrictions on the number of authors.

However, way back at the start of his essay, Greene suggested that those intrepid scientists who could legitimately conduct experiments independently and pull off a sole-author paper would be discouraged (or prevented) from doing so by funding agencies and home institutions in their push for collaborative multidisciplinary teams. This thought was picked up the following March when Kevin Hallock penned a letter in response to suggest that:

funding agencies and institutions should also encourage single-author papers. The effort and initiative required to publish alone suggests an independent and tenacious scientist — both highly desirable qualities in any researcher.

If the ICMJE authorship rules were strictly enforced, many members of this species might well enter the literature as sole authors. Lone authors. Armies of one. But perhaps that would be interpreted as not being collegial or collaborative rather than as being independent and tenacious.

Comments (4)

By JoVE – They’re Indexed!

August 20, 2008 at 12:39 pm · Filed under Biomedical Writing/Editing, Research Resource

The National Library of Medicine will index in PubMed videos from the Journal of Visualized Experiments (JoVE), an online open-access research journal that “publishes” videos of experiments and protocols “to increase reproducibility and transparency in biological sciences.”

Check out the official JoVE blog to learn more about the journal and how you can contribute to their blogging and visualization efforts. The editors also invite you to bring your own blog posts to life with their videos (just ask).

Comments (1)

Older Posts »