Posted on Facebook by a girl I went to high school with. She is a doctor and this was a post by one of the doctors she works with.
DOES HYDROXYCHLOROQUINE HELP IN COVID-19?
I’m writing this in response to seeing a number of people reposting a recent article, “Treatment with hydroxychloroquine, azithromycin, and combination in patients hospitalized with COVID-19,” published online July 1, 2020 by the International Journal of Infectious diseases. The study reported a decreased mortality (death rate) in patients given hydroxychloroquine.
As an emergency physician, I frequently read and listen to critical reviews of the medical literature. I was aware of the apparently positive IJID paper from the Henry Ford system in Southern Michigan and just spent a couple of hours reviewing it. It is unique as so many other studies are unfortunately finding NO BENEFIT FROM HYDROXYCHLOROQUINE (HCQ). The IJID paper has deep methodologic flaws, which is why most physicians and researchers don’t trust the results. You can trust me, or if you’re interested, I’ll break it down in a lot of detail below…
The first red flag is that the article was published as a “pre-proof.” This means it is not the final version of the paper and has not gone through the usual peer review process to prevent publication of studies with significant flaws. This is not usually allowed but because of the fast-moving, urgent need for treatments for Covid-19, some journals are publishing unreviewed articles online, then revising them later after full peer review.
The most important issue is that this is not a prospective randomized control trial, the type of study and level of evidence that we usually use to determine if a drug is effective in treating a disease. Instead, it is an observational retrospective chart review, which can only establish an association between a treatment and an outcome, not determine if a treatment caused the outcome. In this type of study, any number of confounding factors besides the drug in question can cause a difference in the outcome between the groups of patients studied. For there to be any level of confidence that the difference is due to the drug, all other factors should be very similar between the groups.
Instead of randomizing, based on early reports that HCQ and/or azithromycin (AZM) might benefit Covid-19 patients, Henry Ford system created an algorithm to treat 2541 patients with either HCQ + AZM, HCQ alone, or AZM alone or neither. These were not given randomly to similar patients with similar severity of illness, but rather by intention the algorithm used clinical factors to determine who got which treatment. As a result, and as you would expect, there are significant differences in age, comorbidities, other treatments given and pre-treatment disease severity between those getting HCQ and those that did not.
The primary finding in study’s results is a mortality of 13.5% in the HCQ alone group, but 26.4% in the neither medication group. That sounds great, but we need to examine those groups to see if they are otherwise similar. If they are it would suggest HCQ may be effective and should be studied in a randomized trial. If they are dissimilar, it is very likely that other confounders may explain the mortality difference.
We know that Covid-19 mortality increases with increasing age. In the study the median age of those receiving HCQ alone was 53, whereas the median age of those receiving neither was 71. Put another way, only 51% of those receiving HCQ alone were >65 years old, whereas 64% of those receiving neither were age>65. This could explain the entire mortality difference.
In addition, steroids now appear to benefit the sickest Covid-19 patients. In this study 79% of the HCQ alone patients got steroids, whereas only 38% of the neither medication group got steroids. This also could explain the entire mortality difference.
Other differences between the groups include race and BMI. 51% of black patients compared with only 38% of white patients received HCQ alone. Average BMI of those receiving HCQ alone was 31.9, compared with 28 in the neither group. In a well performed randomized control trial, all of these characteristics would be very similar between groups.
The authors attempt a statistical method to account for these differences call propensity-matching, where each patient given HCQ is matched to a very similar patient that didn’t receive HCQ, then these 2 groups are compared. However, out of the 1985 patients that received HCQ they were only able to find 190 similar patients that didn’t receive HCQ for comparison. Typically 80-90% of patients in this type of analysis are matched, not 10%, which further demonstrates how fundamentally different the 2 patient populations were.
It’s hard to believe that these groups of Covid-19 patients that are different in age, race and BMI, received very different rates of steroids, and were sorted based on a clinical algorithm rather than randomized, would have had a similar outcome if neither had been given HCQ. Therefore, it’s impossible to know if the HCQ or all of these other factors is the reason for the difference in mortality.
(Credit: In addition to reviewing the paper myself, many of these points were made in the Emergency Medicine Reviews and Perspectives’ video review by Drs. Dave Schriger and Mel Herbert.)
Here is a list of other studies on HCQ in Covid-19:
United States/Canadian RCT of Hydroxychloroquine for Patients with Early, Nonsevere COVID-19 Finds No Benefit.
Hydroxychloroquine in nonhospitalized adults with early COVID-19: a randomized trial [published online July 16, 2020]. Ann Intern Med. 2020. https://www.acpjournals.org/doi/10.7326/M20-4207
Postexposure Prophylaxis with Hydroxychloroquine Is Not Effective.
A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19 [published online June 3, 2020]. N Engl J Med. 2020. https://www.nejm.org/doi/full/10.1056/NEJMoa2016638
Neither Hydroxychloroquine nor Azithromycin Are Associated with Decreased In-Hospital Mortality in New York.
Association of treatment with hydroxychloroquine or azithromycin with in-hospital mortality in patients with COVID-19 in New York state [published online May 11, 2020]. https://jamanetwork.com/journals/jama/fullarticle/2766117
Hydroxychloroquine Ineffective in Oxygen-Requiring Patients in France.
Mahévas M, Tran V-T, Roumier M, et al. Clinical efficacy of hydroxychloroquine in patients with covid-19 pneumonia who require oxygen: observational comparative study using routine care data. BMJ. 2020;369:m1844.
Hydroxychloroquine Ineffective in Patients with Mild or Moderate COVID-19 in China.
Tang W, Cao Z, Han M, et al. Hydroxychloroquine in patients with mainly mild to moderate coronavirus disease 2019: open label, randomised controlled trial. BMJ. 2020;369:m1849
Hydroxychloroquine Lacks Benefit for COVID-19 in a Large New York City Hospital.
Geleris J, Sun Y, Platt J, et al. Observational study of hydroxychloroquine in hospitalized patients with Covid-19 [published online May 7, 2020; updated May 14, 2020]. N Engl J Med. 2020. https://www.nejm.org/doi/full/10.1056/NEJMoa2012410
Hydroxychloroquine Not Effective for COVID-19 in U.S. Veterans.
Magagnoli J, Narendran S, Pereira F, et al. Outcomes of hydroxychloroquine usage in United States veterans hospitalized with Covid-19 [updated online April 23, 2020]. https://www.medrxiv.org/content/10.1101/2020.04.16.20065920v2
Hydroxychloroquine Not Effective in Randomized Trial in China
Tang W, Cao Z, Han M, et al. Hydroxychloroquine in patients with COVID-19: an open-label, randomized, controlled trial. MedRxiv. 2020 [published online April 14, 2020]. https://doi.org/10.1101/2020.04.10.20060558
DOES HYDROXYCHLOROQUINE HELP IN COVID-19?
I’m writing this in response to seeing a number of people reposting a recent article, “Treatment with hydroxychloroquine, azithromycin, and combination in patients hospitalized with COVID-19,” published online July 1, 2020 by the International Journal of Infectious diseases. The study reported a decreased mortality (death rate) in patients given hydroxychloroquine.
As an emergency physician, I frequently read and listen to critical reviews of the medical literature. I was aware of the apparently positive IJID paper from the Henry Ford system in Southern Michigan and just spent a couple of hours reviewing it. It is unique as so many other studies are unfortunately finding NO BENEFIT FROM HYDROXYCHLOROQUINE (HCQ). The IJID paper has deep methodologic flaws, which is why most physicians and researchers don’t trust the results. You can trust me, or if you’re interested, I’ll break it down in a lot of detail below…
The first red flag is that the article was published as a “pre-proof.” This means it is not the final version of the paper and has not gone through the usual peer review process to prevent publication of studies with significant flaws. This is not usually allowed but because of the fast-moving, urgent need for treatments for Covid-19, some journals are publishing unreviewed articles online, then revising them later after full peer review.
The most important issue is that this is not a prospective randomized control trial, the type of study and level of evidence that we usually use to determine if a drug is effective in treating a disease. Instead, it is an observational retrospective chart review, which can only establish an association between a treatment and an outcome, not determine if a treatment caused the outcome. In this type of study, any number of confounding factors besides the drug in question can cause a difference in the outcome between the groups of patients studied. For there to be any level of confidence that the difference is due to the drug, all other factors should be very similar between the groups.
Instead of randomizing, based on early reports that HCQ and/or azithromycin (AZM) might benefit Covid-19 patients, Henry Ford system created an algorithm to treat 2541 patients with either HCQ + AZM, HCQ alone, or AZM alone or neither. These were not given randomly to similar patients with similar severity of illness, but rather by intention the algorithm used clinical factors to determine who got which treatment. As a result, and as you would expect, there are significant differences in age, comorbidities, other treatments given and pre-treatment disease severity between those getting HCQ and those that did not.
The primary finding in study’s results is a mortality of 13.5% in the HCQ alone group, but 26.4% in the neither medication group. That sounds great, but we need to examine those groups to see if they are otherwise similar. If they are it would suggest HCQ may be effective and should be studied in a randomized trial. If they are dissimilar, it is very likely that other confounders may explain the mortality difference.
We know that Covid-19 mortality increases with increasing age. In the study the median age of those receiving HCQ alone was 53, whereas the median age of those receiving neither was 71. Put another way, only 51% of those receiving HCQ alone were >65 years old, whereas 64% of those receiving neither were age>65. This could explain the entire mortality difference.
In addition, steroids now appear to benefit the sickest Covid-19 patients. In this study 79% of the HCQ alone patients got steroids, whereas only 38% of the neither medication group got steroids. This also could explain the entire mortality difference.
Other differences between the groups include race and BMI. 51% of black patients compared with only 38% of white patients received HCQ alone. Average BMI of those receiving HCQ alone was 31.9, compared with 28 in the neither group. In a well performed randomized control trial, all of these characteristics would be very similar between groups.
The authors attempt a statistical method to account for these differences call propensity-matching, where each patient given HCQ is matched to a very similar patient that didn’t receive HCQ, then these 2 groups are compared. However, out of the 1985 patients that received HCQ they were only able to find 190 similar patients that didn’t receive HCQ for comparison. Typically 80-90% of patients in this type of analysis are matched, not 10%, which further demonstrates how fundamentally different the 2 patient populations were.
It’s hard to believe that these groups of Covid-19 patients that are different in age, race and BMI, received very different rates of steroids, and were sorted based on a clinical algorithm rather than randomized, would have had a similar outcome if neither had been given HCQ. Therefore, it’s impossible to know if the HCQ or all of these other factors is the reason for the difference in mortality.
(Credit: In addition to reviewing the paper myself, many of these points were made in the Emergency Medicine Reviews and Perspectives’ video review by Drs. Dave Schriger and Mel Herbert.)
Here is a list of other studies on HCQ in Covid-19:
United States/Canadian RCT of Hydroxychloroquine for Patients with Early, Nonsevere COVID-19 Finds No Benefit.
Hydroxychloroquine in nonhospitalized adults with early COVID-19: a randomized trial [published online July 16, 2020]. Ann Intern Med. 2020. https://www.acpjournals.org/doi/10.7326/M20-4207
Postexposure Prophylaxis with Hydroxychloroquine Is Not Effective.
A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19 [published online June 3, 2020]. N Engl J Med. 2020. https://www.nejm.org/doi/full/10.1056/NEJMoa2016638
Neither Hydroxychloroquine nor Azithromycin Are Associated with Decreased In-Hospital Mortality in New York.
Association of treatment with hydroxychloroquine or azithromycin with in-hospital mortality in patients with COVID-19 in New York state [published online May 11, 2020]. https://jamanetwork.com/journals/jama/fullarticle/2766117
Hydroxychloroquine Ineffective in Oxygen-Requiring Patients in France.
Mahévas M, Tran V-T, Roumier M, et al. Clinical efficacy of hydroxychloroquine in patients with covid-19 pneumonia who require oxygen: observational comparative study using routine care data. BMJ. 2020;369:m1844.
Hydroxychloroquine Ineffective in Patients with Mild or Moderate COVID-19 in China.
Tang W, Cao Z, Han M, et al. Hydroxychloroquine in patients with mainly mild to moderate coronavirus disease 2019: open label, randomised controlled trial. BMJ. 2020;369:m1849
Hydroxychloroquine Lacks Benefit for COVID-19 in a Large New York City Hospital.
Geleris J, Sun Y, Platt J, et al. Observational study of hydroxychloroquine in hospitalized patients with Covid-19 [published online May 7, 2020; updated May 14, 2020]. N Engl J Med. 2020. https://www.nejm.org/doi/full/10.1056/NEJMoa2012410
Hydroxychloroquine Not Effective for COVID-19 in U.S. Veterans.
Magagnoli J, Narendran S, Pereira F, et al. Outcomes of hydroxychloroquine usage in United States veterans hospitalized with Covid-19 [updated online April 23, 2020]. https://www.medrxiv.org/content/10.1101/2020.04.16.20065920v2
Hydroxychloroquine Not Effective in Randomized Trial in China
Tang W, Cao Z, Han M, et al. Hydroxychloroquine in patients with COVID-19: an open-label, randomized, controlled trial. MedRxiv. 2020 [published online April 14, 2020]. https://doi.org/10.1101/2020.04.10.20060558