As more studies roll in, little evidence that hydroxychloroquine works

A study observing COVID-19 patients has found no evidence that the malaria drug hydroxychloroquine, touted as a possible treatment for COVID-19, made a difference to the chance that patients would need a ventilator. The results also suggested that patients treated with hydroxychloroquine had a higher rate of death than those who weren’t treated with the drug.

The study was not a randomized clinical trial, which means that the evidence it offers is tentative and should be interpreted with caution. It was also published on preprint server medRxiv, which means it has not yet been peer-reviewed.

But interpreting the evidence with caution does not mean disregarding it completely. This study is one of a growing number telling us that we don’t yet know enough about hydroxychloroquine, adding more weight to the argument that we need to wait for better-quality evidence from randomized controlled trials before we start widespread use of a drug with significant side effects.

Premature

Some small studies have given us reasons to think that chloroquine and hydroxychloroquine could have potential as treatments for COVID-19. In some cases, the findings come from experiments in cultured cells, which won’t necessarily translate directly to using the drugs in sick humans. In others, the findings come from small studies that have critical flaws like using very small groups of patients, having no control group, or excluding patients who died from analysis.

“Normally, such research would be deemed hypothesis-generating at best,” wrote doctors Jinoos Yazdany and Alfred Kim in an opinion piece in the Annals of Internal Medicine. And they were released at the same time that other early studies were finding no evidence that these drugs help COVID-19 patients.

However, early hype—including repeated promotion from President Trump—led to a runaway train of enthusiasm for the drugs. The Food and Drug Administration authorized treatment of COVID-19 patients with chloroquine and hydroxychloroquine despite the lack of good evidence for their efficacy, sparking backlash from former FDA leaders.

These drugs have a range of possible adverse effects, including serious cardiac damage. Using them for critically ill COVID-19 patients therefore not only runs the risk of not helping, but also of actively harming people. The worldwide run on the drugs and resulting shortages are also a problem for patients using them for conditions like lupus, where they have been found to be effective.

A range of clinical trials are now underway to establish whether these drugs are actually beneficial. In the meantime, the FDA authorization means that there is a growing pile of data from patients who have been treated with them.

Testing on the fly

The US Veterans Health Administration is a national system of clinics, hospitals and other medical centers. Because it’s a single organization, data on patients is gathered in a consistent way, which makes it easier for researchers to compare apples with apples.

A team of researchers used VHA data to track the outcomes of confirmed COVID-19 patients at veterans hospitals who were treated with just hydroxychloroquine, hydroxychloroquine plus an antibiotic, or neither of the drugs. They found that 27.8 percent of the 97 patients treated with just hydroxychloroquine died, compared to 11.4 percent of the 158 patients who weren’t treated with hydroxychloroquine at all, and 22.1 percent of the 113 patients who were treated with hydroxychloroquine and an antibiotic. Rates of ventilation were similar across the three groups.

This evidence is weaker than a randomized controlled trial because the patients who were given different treatments may have had other important differences to begin with. In a randomized trial, patients are assigned different treatments (or a placebo) randomly, which means that different groups should all have a roughly similar mix of people who are very sick or only a little bit sick, old and young, and so on.

In a retrospective study like this one, the doctors may have given the hydroxychloroquine treatment only to the sickest patients, in which case we’d expect that group to have worse outcomes. There are ways to try to account for this lack of randomization in the statistical tests that researchers use to calculate the risks across different groups, but these adjustments require the researchers to work out what other factors might complicate the analysis—a difficult challenge with a random population like this one.

The patients also weren’t representative of the wider population. They were all men and all older than 59 years, which means that the results wouldn’t necessarily be the same in younger groups or among women.

The results don’t mean that hydroxychloroquine is definitely useless or that clinical trials should be halted. Recent NIH guidance for clinicians treating COVID-19 patients says that there currently isn’t enough evidence to recommend for or against treating with hydroxychloroquine, and that remains true.

But they do offer more evidence suggesting that we don’t yet know enough to forge ahead with using the drugs as treatment. The authors of the study acknowledge the shortcomings of their own work but argue that the results nonetheless “highlight the importance of awaiting the results of ongoing prospective, randomized, controlled studies before widespread adoption of these drugs.”