Editor's Note: On June 4, Mandeep Mehra and two co-authors wrote to the Lancet and New England Journal of Medicine requesting retraction of the two studies. This article has also been updated to reflect the WHO's decision to resume enrollment in the Solidarity trial.
Dive Brief:
- Two top medical journals have released "expressions of concern" regarding high-profile studies they published of repurposed drugs used to treat patients with COVID-19. The research was based on a hospital records database, the quality of which some scientists and doctors have questioned.
- The two analyses were of COVID-19 patients taking, respectively, the malaria drug hydroxychloroquine or various medicines for high blood pressure. The hydroxychloroquine study found no benefit, and notable harms, to the drug's use, while the other study concluded high blood pressure pills were safe for patients with severe coronavirus infections.
- The studies, and the conclusions drawn in response, illustrate the challenges facing medical research as national leaders and the broader public seek quick answers before more rigorous and controlled trials are able to deliver results.
Dive Insight:
Response to the hydroxychloroquine study, which was published in The Lancet and suggested an increased risk of death among those taking the drug, was swift.
Soon after, the World Health Organization temporarily stopped enrolling patients to receive hydroxychloroquine in its international "Solidarity" study while a data safety monitoring board reviewed the trial and The Lancet findings. On Wednesday, the WHO said it plans on re-starting enrollment following a recommendation from the safety board.
The second analysis, meanwhile, looked at patients who had taken one of two types of drugs used to treat high blood pressure, but which have previously been linked to increased risk of severe infections. The study, published in The New England Journal of Medicine, found no such risk.
But both medical journals have now published "expressions of concern" about the data used to support the studies' conclusions.
Both analyses were retrospective reviews, conducted by Harvard medical professor Mandeep Mehra, of hospital records kept by Surgisphere, a data analytics company. However, some medical researchers have challenged the accuracy of that database in an open letter, pointing to the small number of hospitals that accounted for most of the confirmed coronavirus cases.
The open letter, signed by 174 researchers, points specifically to weaknesses in the data from the U.K., where 706 patients were included in the database but none from a hospital in London, the center of that country's outbreak. Likewise, Turkey's data is flawed, the letter states, because Surgisphere claims that 346 patients had been hospitalized as of March 15 while the country's ministry of health had confirmed only a single coronavirus patient at that time.
An additional critique was that Mehra's analysis of blood pressure drugs found a far lower risk of death among patients who were older than 80 years than had previous work, according to the letter.
The letter asks for more detail on the Surgisphere data and allowing the journals to independently validate it. The Lancet's "expression of concern" stated that an independent audit of the data by most of the authors of the hydroxychloroquine analysis is underway.
In a statement, Surgisphere defended its data, saying: "We are committed to demonstrating the high standards we hold at Surgisphere, and the robustness of the work that has been completed."
With public health officials and policymakers searching for ways to blunt the pandemic, retrospective reviews like these have sought to deliver quick research to help guide treatment. However, their use comes with significant risk of misleading people due to the limitations of data generated by retrospective studies, particularly compared to randomized, controlled trials.
"The conversation and attention to every little study has been hugely damaging," said University of Pittsburgh professor Walid Gellad in an interview with BioPharma Dive. "It's been damaging because every bit of information, not matter what your pre-existing thoughts were about the drug, are magnified and taken to the extreme."
With hydroxychloroquine — use of which has become a political issue in the U.S. — more reliable information could soon be available from randomized, controlled trials now ongoing. Results are expected soon from a study conducted by the University of Minnesota and others in the U.K. and Europe could soon read out data, too.
"One well-run, high quality RCT that has enough people in it would supersede the observational studies," Gellad said.