Update: JAMA has a nice summary of the true impact of COI, even on well-intentioned investigators.
In the same issue of Accountability in Research as the report on the effectiveness of RCR education, Kevin Elliott raises concerns about “Scientific Judgment and the Limits of Conflict-of-Interest Policies.” He notes that “these worries are aggravated by the fact that many universities are currently seeking to bolster their research portfolios and boost local economies by developing more extensive partnerships with private industry.” The abstract of this well-crafted and insightful article says it all:
This article argues that the three major elements of typical university conflict-of-interest (COI) policies (i.e., disclosure, management, and elimination of conflicts via divestiture or recusal) are likely to be insufficient for screening out many worrisome influences of financial COIs. Current psychological research challenges the effectiveness of disclosure, management plans are unlikely to address the wide range of ways that financial COIs can influence scientific judgment, and it is often impractical to eliminate conflicts. Identifying the limits of these policies highlights the importance of considering alternative strategies, such as encouraging more independently funded research, in order to maintain the integrity of science.
Elliott notes that “A much more significant and yet less frequently discussed reason for thinking that disclosure policies are insufficient to address financial COIs is that they may do little or no good even when there is adequate standardization of policies and compliance with them! The assumption behind the recent emphasis on disclosing financial COIs is that this approach mitigates many of the worrisome effects of COIs while causing minimal disruption to the activities of researchers. Ideally, the disclosure requirement allows those who receive information to be on the lookout for worrisome influences, and it provides motivation for scientific researchers to provide untainted information. Unfortunately, a fascinating body of psychological research challenges these hopes for the effectiveness of disclosure policies.”
He goes on to review this “fascinating body of psychological research” … and let’s not gloss over the key word in his comment above: “ideally“. Who on the University payroll is dedicated to staying “on the lookout for worrisome influences” for the hundreds (thousands) of potential COI reported (or to confirm the unreported COI doesn’t exist)?
Although I realize the NIH cannot take on the burden of actually managing the financial COI reported to them, surely the Roadmap needs some sort of COI GPS to keep federally funded investigators from wandering off the high road. Perhaps one of the lawyers among us could explain why the NIH’s merely collecting “details regarding the nature of financial conflicts of interest and how they are managed, reduced, or eliminated” places the burden of enforcement on the NIH (versus simply having the data on hand for audits/spot checks and providing a mechanism for ensuring their grantee institutions truly do manage COI in ways that go beyond collecting a PI’s signature on a form that is sent to filing cabinet purgatory).
Going back to the RCR article, the slipperiness of identifying and managing COI becomes clear in this comment made by a grad student after attending the lecture:
I: In your mind, what level of financial gain would raise the risk of conflict of interest to an unacceptable level?
P: This was kind of interesting when they talked about it in class, but for people making different amounts of money, it seems to be quite a varying scale probably. I guess that at the point I amit wouldn’t take very much. It would take maybe $1000, but I think later, when I’m making more money, that level would probably get raised a little bit at least. But I think even if something is as simple as $1000 even if you’re making $100,000 per year, you still might have things going on a little bit where you’re not totally subjective with what you’re doing. (Ph.D. student with no prior RCR training)
Not sure if the FASEB toolkit and proposals for Conflict Vitae can solve these issues, especially as research teases out how COI may play out in the brain.
Indeed, Elliott focuses on COI management as the neglected stepchild of this process. After all, it is easy and feels good (righteous) to check boxes and sign a form – but then what? At the heart of the matter, “Efforts to manage COIs (e.g., via supervision by internal or external committees) are likely to be either limited in effectiveness or extremely bureaucratic and time-consuming, because there are so many ways in which judgments that might be influenced by COIs permeate scientific research, … there appears to be sufficiently widespread potential for serious influences of COIs (in at least some scientific fields) that it is unrealistic to think that universities could carefully manage all studies or scientists where the influences might be present.”
Points of management suggested by Elliott include the judgment of university scientists (serving the public interest), the choice of language, the methods used to analyze and interpret data, and the dissemination of results.
He summarizes the likely major responses to his findings as “dismissing the significance of at least some ways in which financial COIs influence research [status quo]; relying on and perhaps strengthening existing mechanisms that promote criticism and deliberation within the scientific community [stepped up self-policing]; developing new strategies in addition to the components of current COI policies in an effort to alleviate questionable influences [“Science Court”, independent testing agency]; and preventing or eliminating COIs in more cases than universities might otherwise consider [just say no to bad $].”
Which leads us to the final component of the recent hat-trick of COI news items: the NEJM article on selective reporting of antidepressant trials. Published reports (n=37) suggest that 94% of trials conducted were positive, while looking at all reviews submitted to the FDA (n=74) put the rate at 51%, with an overall success rate of 32%. Now, COI is not an issue discussed in this report, and the main ethical concern rightly raised is the lack of respect for study volunteers who put themselves at risk for the greater good. Nonpublication of negative results is common and occurs for many reasons, but the possibility of COI cannot be ignored, particularly when psychiatry receives more pharma bennies than any other specialty.
Among his suggestions, Elliott notes that “attention to the limitations of disclosure and management might encourage university administrators to consider preventing or removing COIs in more situations. Administrators might also think twice about establishing particularly aggressive institutional links with industry …” Of course, I cannot help but again revisit what I feel is the poster child of morally and ethically bankrupt university-industry linkages in which COI (not just financial) permeates all aspects of the relationship: academic medical centers partnering with the tobacco industry.