Ivermectin for COVID-19: Addressing Potential Bias and Medical Fraud

Abstract

Ivermectin has become a controversial potential medicine for coronavirus disease 2019. Some early studies suggested clinical benefits in treatment of infection. However, the body of evidence includes studies of varying quality. Furthermore, some trials have now been identified as potentially fraudulent. We present a subgroup meta-analysis to assess the effects of stratifying by trial quality on the overall results. The stratification is based on the Cochrane Risk of Bias measures and raw data analysis where possible. The results suggest that the significant effect of ivermectin on survival was dependent on largely poor-quality studies. According to the potentially fraudulent study (risk ratio [RR], 0.08; 95% CI, 0.02–0.35), ivermectin improved survival ~12 times more in comparison with low-risk studies (RR, 0.96; 95% CI, 0.56–1.66). This highlights the need for rigorous quality assessments, for authors to share patient-level data, and for efforts to avoid publication bias for registered studies. These steps are vital to facilitate accurate conclusions on clinical treatments.

COVID-19, SARS-CoV-2, ivermectin
Topic: fraud, ivermectin, covid-19
Issue Section: Perspectives

In June 2020, ivermectin, a Food and Drug Administration (FDA)–approved antiparasitic drug, was shown to have antiviral effects against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in vitro [1]. Following this, ~86 clinical trials investigating ivermectin for coronavirus disease 2019 (COVID-19) have been registered globally. In late 2020, clinical trials began reporting very compelling clinical benefits for ivermectin in the treatment of COVID-19. From late 2020 onwards, multiple groups produced meta-analyses that reported that ivermectin had a significant effect on survival, hospitalizations, clinical recovery, and viral clearance [2, 3]. Our meta-analysis was first presented in January 2021 and published in July 2021 [4]. It suggested that ivermectin resulted in a significant 56% improvement in survival, favorable clinical recovery, and reduced hospitalizations. Such optimistic results from multiple meta-analyses have escalated public interest in using ivermectin for the treatment and prevention of COVID-19, despite the World Health Organization (WHO) only recommending its use within clinical trials [5, 6].

However, as with all meta-analyses, a key limitation is the quality and completeness of the available evidence. During our original assessment of studies, standardized Cochrane Risk of Bias measures (RoB 2) had classified several studies as “high risk of bias” [7]. A study by Niaee et al. from Iran that reported a randomized methodology was found to have significant differences in baseline characteristics across treatment arms [8]. This suggests that the participants were not randomized appropriately, which could bias the outcomes. Second, a study by Okumus et al. from Turkey did not provide any information on allocation concealment, and it was unclear if the participants or investigators were blinded, which risks introducing observation bias [9]. Lastly, a study by Hashim et al. from Iraq provided insufficient details on the randomization process, had a lack of clarity on participants who were analyzed, and involved unblinded assessment of a subjective outcome [10].

Furthermore, some studies were then identified to be potentially fraudulent. For example, on July 15, 2021, a study by Elgazzar et al. from Egypt was retracted from the preprint server Research Square due to “ethical concerns” [11]. It has been reported that the data for ~79 participants were duplicates, some deaths were recorded on dates before the trial had started, and instances of plagiarism were identified in the text [12]. Similarly, a study conducted in Lebanon by Raad et al. was also identified to have duplicate data for multiple participants when the patient-level database was analyzed in September 2021 [13]. Before these inconsistencies were identified, the Elgazzar and Raad studies had been included in multiple meta-analyses, which suggested significant benefits for ivermectin in the treatment of COVID-19 [2, 3]. In our original meta-analysis, the Raad study accounted for 11.8% of the effect of ivermectin on hospitalization, and the Elgazzar study accounted for 12.6% of the effect of ivermectin on survival [4].

These instances suggest that the data available to the support the use of ivermectin for COVID-19 are not reliable. In July 2021, after the potentially fraudulent studies were identified, we retracted our published meta-analysis and began working on an updated analysis assessing the effects of stratifying by trial quality on the overall results. Clinical trials evaluating ivermectin for the treatment of COVID-19 had been identified by systematic searching of 8 databases. An in-depth evaluation of study quality was conducted, in addition to the standard Cochrane RoB 2 and CONSORT checklist [7]. First, we evaluated trials based on the effectiveness of their randomization process by comparing baseline characteristics across treatment arms using the chi-square test. Second, randomization dates were checked to ensure that patients were randomized into the treatment arms on similar dates. Third, checks were conducted to evaluate if recruitment to treatment arms was balanced at each investigational center. Furthermore, we analyzed patient-level databases, where available, to check for any evidence of duplicate participants and unexpected homogeneity or heterogeneity. From this, a meta-analysis was conducted with subgroups of clinical trials at different risk of bias levels.

Looking at the key survival outcome, the analysis includes 12 studies with a total of 2628 participants (Table 1) [4]. This included 4 studies at a low risk of bias, 4 with some concerns, 3 at a high risk of bias, and 1 potentially fraudulent study. The analysis demonstrates that on including all 12 studies, ivermectin results in a significant 51% increase in survival (95% CI, 0.28–0.86; P = .01) (Figure 1; Supplementary Figure 1). On excluding the potentially fraudulent Elgazzar study, ivermectin results in a borderline significant 38% increase in survival (95% CI, 0.39–0.99; P = .05) (Figure 1; Supplementary Figure 2). On excluding the high risk of bias studies, ivermectin results in a nonsignificant 10% increase in survival (95% CI, 0.57–1.42; P = .66) (Figure 1; Supplementary Figure 3). Lastly, on excluding studies with some concerns of bias, ivermectin results in a nonsignificant 4% increase in survival (95% CI, 0.56–1.66; P = .90) (Figure 1; Supplementary Figure 4). These observations demonstrate that the significant effect of ivermectin on survival was dependent on the inclusion of studies with a high risk of bias or potential medical fraud.

Table 1.Studies Included in the Survival Analysis [4]
Study Risk of Bias Level Sample Size Ivermectin Arm Control Arm
Lopez-Medina et al. Low risk 398 0/200 1/198
Fonseca et al. Low risk 168 12/53 25/115
Zoni et al. Low risk 501 4/250 3/251
Kirti et al. Low risk 112 0/55 4/57
Rezai et al. Some concerns 69 1/35 0/34
Abd-Elsalam et al. Some concerns 164 3/82 4/82
Gonzalez et al. Some concerns 73 5/36 6/37
Mahmud et al. Some concerns 363 0/183 3/180
Niaee et al. High risk 180 4/120 11/60
Hashim et al. High risk 140 2/70 6/70
Okumus et al. High risk 60 6/30 9/30
Elgazzar et al. Apparent fraud 400 2/200 24/200
Total 2628 39/1314 96/1314
Open in new tab
Figure 1.
Survival effects of ivermectin.
Open in new tabDownload slide
Survival effects of ivermectin.

There are added challenges with clinical trials investigating COVID-19 treatments. In a rapid response to COVID-19, many small-scale studies have been conducted globally for potential agents. However, not all trials have reported findings. An example is a trial for nitazoxanide in Brazil with 600 participants that was completed in October 2020 but has not reported any results yet [14]. Publication bias impacts meta-analyses, with positive and significant results more likely to be reported. Therefore, we believe that by including publication bias, it is even less likely that ivermectin will show significant benefits for COVID-19 treatment. Some nonrandomized trials may also be overinterpreted. For example, in a nonrandomized retrospective cohort study, remdesivir demonstrated an improvement in clinical recovery and reduced mortality risk by 62% [15]. However, when evaluated in the WHO’s randomized placebo-controlled SOLIDARITY trial, remdesivir had little to no effect on mortality [16]. Any initial promising findings from a small number of sources need to be interpreted with caution, studied further, and considered within the wider body of evidence.

The results from this analysis highlight the need for rigorous quality assessments when evaluating clinical trials of drugs for COVID-19. Existing and widely used risk of bias assessment tools are not enough. These tools provide a systematic framework for identifying potential key sources of bias in a trial’s internal methodology but work on the fundamental assumption that a published study is reporting accurate and complete findings. They allow reviewers to make judgments on the assumption that basic standard procedure is followed, the data are real, and no information is being intentionally hidden.

With cases of potential medical fraud now identified, it is essential that access to patient-level databases be provided. If authors fail to provide these data, the study should be considered with a higher index of suspicion. Additionally, it should be mandatory that all registered trials report their findings. We understand that these are substantial changes to established procedures. However, the failure to recognize the potentially fraudulent studies, which led to multiple meta-analyses suggesting significant benefits of ivermectin for COVID-19, indicates that the tools currently used to evaluate the quality of clinical trials are insufficient. These events warrant our stringent recommendations.

Financial support. This work was funded by the Rainwater Charitable Foundation.

Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Patient consent. All the clinical trials included in the meta-analysis were approved by local ethics committees, and all patients gave informed consent.

Acknowledgments
We would like to thank Gideon Meyerowitz-Katz, Kyle A. Sheldrick, and Jack M. Lawrence for their help with the quality assessments.

References
1. Caly L, Druce J, Catton M, Jans D, Wagstaff K. The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro. Antiviral Res 2020; 178:104787.
Google ScholarCrossrefPubMed
2. Bryant A, Lawrie TA, Dowswell T, et al. . Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. In press.
Google Scholar
3. Kory P, Meduri GU, Varon J, Iglesias J, Marik PE. Review of the emerging evidence demonstrating the efficacy of ivermectin in the prophylaxis and treatment of COVID-19. Am J Ther 2021; 28:e299–318.
Google ScholarCrossrefPubMed
4. Hill A, Garratt A, Levi J, et al. . Meta-analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection. Open Forum Infect Dis 2021; XXX:XXX–XX.
Google Scholar
5. Robins-Early N. Desperation, misinformation: how the ivermectin craze spread across the world. The Guardian. 24 September 2021. Available at: https://www.theguardian.com/world/2021/sep/24/ivermectin-covid-peru-misinformation Accessed 15 October 2021.
6. World Health Organization. Therapeutics and COVID-19: living guideline 2021. 2021. Available at: https://www.who.int/publications/i/item/WHO-2019-nCoV-therapeutics-2021.2. Accessed 13 October 2021.
7. Sterne JAC, Savović J, Page MJ, et al. . RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366:l4898.
Google ScholarPubMed
8. Gheibi N, Shakhsi Niaee M, Namdar P, et al. . Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: a randomized multi-center clinical trial. Asian Pac J Trop Med 2021; 14:266.
Google ScholarCrossref
9. Okumuş N, Demirtürk N, Çetinkaya R, et al. . Evaluation of the effectiveness and safety of adding ivermectin to treatment in severe COVID-19 patients. BMC Infect Dis. In press.
Google Scholar
10. Hashim H, Maulood M, Rasheed A, Fatak D, Kabah K, Abdulamir A. Controlled randomized clinical trial on using ivermectin with doxycycline for treating COVID-19 patients in Baghdad, Iraq. medRxiv 2020.10.26.20219345 [Preprint]. 27 October 2020. Available at: https://doi.org/10.1101/2020.10.26.20219345. Accessed 13 October 2021.
11. Elgazzar A. Notice of retraction: Efficacy and safety of ivermectin for treatment and prophylaxis of COVID-19 pandemic. Res Square rs-100956 [Preprint]. 16 November 2021. Available at: https://assets.researchsquare.com/files/rs-100956/v2/c11416a2-d0bd-494f-abc8-3cbf8c605b10.pdf?c=1631861037. Accessed 13 October 2021.
Google Scholar
12. Reardon S. Flawed ivermectin preprint highlights challenges of COVID drug studies. Nature 2021; 596:173–4.
Google ScholarCrossrefPubMed
13. Samaha A, A, Mouawia H, et al. . Effects of a single dose of ivermectin on viral and clinical outcomes in asymptomatic SARS-CoV-2 infected subjects: a pilot clinical trial in Lebanon. Viruses. In press.
Google Scholar
14. Nitazoxanide therapy for patients with COVID-19 pneumonia – full text view – ClinicalTrials.gov. 2021. Available at: https://clinicaltrials.gov/ct2/show/NCT04561219?cond=nitazoxanide&draw=2&rank=6 Accessed 13 October 2021.
15. Gilead. Gilead presents additional data on investigational antiviral remdesivir for the treatment of COVID-19 2020. 2020. Available at: https://www.gilead.com/news-and-press/press-room/press-releases/2020/7/gilead-presents-additional-data-on-investigational-antiviral-remdesivir-for-the-treatment-of-covid-19. Accessed 13 October 2021.
16. World Health Organization. Repurposed antiviral drugs for Covid-19 — interim WHO solidarity trial results. N Engl J Med 2020; 384:497–511.
PubMed
© The Author(s) 2022. Published by Oxford University Press on behalf of Infectious Diseases Society of America.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

Writing about COVID-19

As well as building up a resource of information and analysis on COVID-19, we want to ensure that we pass on any tips about what can go wrong when writing about this subject: and how to get it right! If you have experience in writing about this area and feel you have advice that would help others, please contact us at: covid19editor@wfsj-briefing.org.

We’d also like to hear from you at that address if you would like to regularly contribute links to the site. If you just want to suggest links on an occasional basis, please send them to: covid19@wfsj-briefing.org.