Do big name journals have a 'marketing trial' problem?
Clinical trials published in high-impact journals such as the Lancet or the New England Journal of Medicine (NEJM) have recently come under scrutiny.
A group of editors from the BMJ and PLOS Open Medicine sought to quantify how many clinical trials in respected, peer-reviewed medical journals could be classified as "marketing trials." While the editors noted the difficulty in determining the true research objective of trials, they endeavored to classify characteristics of trials which appeared to be marketing oriented.
Overall, BMJ editor-in-chief Fiona Godlee and her co-authors determined one-fifth of the original-data clinical trials published in six top medical journals in 2011 could be classified as marketing trials. Notably, they found almost one-third of the trials in NEJM and 23% of the trials of those from the Lancet were marketing trials.
In comparing the groups, the study found those trials classified as marketing had higher rates of funding and participation by the drug manufacturer, as well as a greater median number of trial sites. The abstract detailing the study can be found here.
What is a marketing trial?
A marketing trial is designed principally to highlight data in a manner which portrays a drug in a positive light. This approach to reporting clinical trial data not only goes against the basic premise of a clinical trial---to test a scientific hypothesis---but it also runs afoul of the publication rules at peer-reviewed journals. However, as the study authors note, there are few documented examples of purely marketing trials because it is difficult to ascertain original motives for a study.
Peer-reviewed journals are the gold standard in medical publishing, and therefore associated with rigorous publishing guidelines. Peer-review aims to ensure quality by adhering to specific standards and conventions. Publication in one is important because it gives a study the imprimatur of research integrity and transparency.
How the study was handled
The research team examined 194 randomized clinical trials published in six medical journals over the course of 2011. In addition to the NEJM and the Lancet, the researchers also reviewed studies published in JAMA, the Annals of Internal Medicine, PLOS Medicine, and the BMJ.
The six researchers agreed upon six characteristics suggestive of a marketing trial: a high level of involvement by the drug manufacturer in the study design, data analysis, and reporting of the study (1-3), recruitment of small numbers of patients from multiple study sites for a common disease (4), misleading abstracts (5), and conclusions focusing on secondary endpoints (6).
Each researcher reviewed the trials and rated them as yes (a marketing trial),no (not a marketing trial) or maybe. Whenever four of the researchers rated a trial as "yes," the trial was categorized as a marketing trial. Twenty-one percent were characterized as yes, 72% as no, and only 7% as maybe.
Other blinded researchers then extracted data on the trial characteristics of the studies in each group for comparison.
Marketing versus science
All of the trials rated "yes" and "maybe" were found to be funded by the manufacturer of the drug in question, compared to 37% of the "no" trials. Furthermore, over four-fifths of the "yes" trials had authors or contributors from the manufacturer involved in study design (83%), data analysis (85%), and reporting (81%). This compares unfavorably to the "no" trials, which has rates of 19%, 15%, and 15% respectively.
The researchers also found marketing trials had more clinical trial sites than other clinical trials—a median of 171 sites versus 13 sites. Higher numbers of study sites means more physicians are involved with the product being tested, thereby increasing physician exposure to the product and setting up a potential network of regional key opinion leaders.
"Yes" trials were also more likely to use surrogate endpoints or to focus on composite outcomes, although those endpoints may not be the most relevant to patients.
The good part about marketing trials
Carl Heneghan, Director of the Centre for Evidence-based Medicine in the U.K. was one of the co-authors on The BMJ paper. In subsequent analysis, Heneghan noted “there were some positive features of the marketing trials when compared with non-marketing trials.”
Specifically, marketing trials were more likely to have better reporting with respect to blinding, safety outcomes and adverse events.
Are the researchers biased?
The six researchers acknowledge their classifications were necessarily subjective and upfront about their potential conflicts of interest as employees of some of the journals they were studying.
"In effect, the group of experts through consensus were able to identify a group of suspected marketing trials, but this grouping could not be validated, and remains open to experimenter’s bias: a subjective bias whereby the result is overtly influenced by the experimenters, in this case the six raters," the researchers wrote in their discussion.
The researchers reached out to both the editors of the journals and to the authors of the studies included for response. The editors of these top-tier journals were adamant marketing considerations did not play a role in the studies they chose to publish. They instead focused on the reasons why they might choose to publish a particular study—mainly due to the novelty of the clinical research, or the importance of a specific topic.
Despite the implicit bias in their categorization of trials, the researchers felt this study provided a launching pad for further examination of published peer-review trials.
What makes good data?
Even if a trial could be considered a marketing trial, it doesn't necessarily follow that it has no clinical value. "The question as to whether marketing trials exist remains controversial," wrote the authors.
Some people, such as Marcia Angell a former editor at NEJM, characterize medical journals as mere “advertising agencies for pharmaceutical companies,” but this isa decidedly minority view.
Clinical data also plays a central role in pharmaceutical marketing, because as the researchers noted “an absence of marketing may undermine implementation of beneficial interventions.”
Despite the potential flaws of this study, it raises an important question over how the medical community should vet and value widely published studies. Ensuring trial and publication integrity is vital to advancing the science being studied.