The degree of failure in this study is not an unfortunate scientific event. It is a terrible failure, reports Simon Collins at HIV i-Base.
Simon Collins, HIV i-Base
On 5 June 2020, the large randomised RECOVERY study announced that hydroxychloroquine (HCQ) will no longer be used to treat COVID-19. [1]
The results show that hundreds of people died – both taking HCQ and in the comparison group receiving no investigational drugs – and yet the study was only closed because of a safety request by the UK Medicines and Healthcare products Regulatory Agency (MHRA).
This very large study – with more than 11,000 participants – should have been looking for early signals that experimental treatment might be effective. Instead, the results announced yesterday call for an urgent analysis of other ongoing arms plus immediate use of drugs that are now known to be effective.
The press release states that people receiving HCQ did no better than people who were given no additional drugs – also called the standard of care. Actually, it is not that HCQ didn’t make enough difference for the results to be significantly better. It looks like people taking HCQ did worse than adding nothing, at least numerically. It is very worrying that this analysis was only carried out because of a request by the MHRA.
For the main study endpoint – the number of people still alive after 28 days – approximately 25.7% of the 1542 people who received HCQ died compared to 23.5% of the 3132 people who were given no treatment. In a study this large, this signal of no active benefit from an experimental treatment should have been found much earlier. There were also apparently no benefit in other measures, such as length of hospital stay or other clinical factors.
How many deaths were these researchers going to allow to continue before they would have stopped the study themselves? Almost 400 people in the HCQ arm and more than 700 people in the control arm died and the study still planned to continue?
Many researchers will not be surprised at the lack of effect with HCQ – but they should be shocked at the time taken to hear this result. Over the last few weeks, other studies have been published showing that HCQ was unlikely to work. [2-6]
One of these – a very large study published in the Lancet reported that HCQ was not effective, and prompted the RECOVERY study to look at their own results. And they came back saying their study should continue unchanged. [7]
Anyone reading that letter from the RECOVERY researchers on 22 May 2020, would expect results to perhaps already show a trend towards benefit from HCQ. This would support continuing to allow participants to take a risk until the study reached a conclusion that was statistically significant.
Instead, nothing close to a benefit could have been happening. It doesn’t even matter that the Lancet study has since been retracted – another complicated story [8, 9] – the important thing is that three weeks ago the RECOVERY study insisted that their data supported continuing to use HCQ.
Today’s results showing no suggestion of benefit are important for several reasons.
Firstly, the large RECOVERY study is continuing with other single therapy arms, some of which have even less evidence than HCQ to show they might work. These include monotherapy (single drug) using an old HIV drug called lopinavir/r (LPV/r). Actually, in March 2020, an earlier randomised study was published showing no benefit from lopinavir/r. [10] For RECOVERY to be continuing with this drug, it needs to already be showing a strong trend towards benefit. Anything less, and LPV/r should also be pulled like HCQ. The RECOVERY study should not be looking for small marginal benefits, but for clear signals that the experimental drugs are considerably better than standard of care.
RECOVERY is also studying a single antibiotic called azithromycin (AZM). Actually, previous studies claiming a benefit from HCQ used it together with AZM. This does not make it plausible that AZM monotherapy will be a success. Again, anything less than clear benefit compared to standard of care, and this arm should be pulled too. And with a study this large, the results should have been available weeks ago. This shouldn’t need a prompt letter from the MHRA over safety. The DSMB in the RECOVERY study, should be analysing every death, with a low threshold of benefit to continue and a similarly low threshold to stop.
Secondly, the statistical plans and timeline for analysing early results should be part of the RECOVERY study protocol – as it is for other major studies. The protocol should publish the start/stop criteria and the thresholds that are being used. It is not good enough for the study to say, as it currently does, that it will look at the results every two weeks. If this is the case, why has it taken an MHRA directive to look again now?
Thirdly, we need to remember the context for COVID-19. Large numbers of participants who are already hospitalised have trusted the researchers to take experimental drugs that have some chance of working (and the chance of no drugs). By joining RECOVERY, people are by default not joining another study – and the UK already has another 20 or so ongoing treatment trials. [11]
Finally, on 26 May 2020, the new availability of remdesivir, a proven treatment for COVID-19, should have led this to be offered to all participants in RECOVERY. Based on published results from the large randomised ACTT study showing remdesivir to be effective, the MHRA announced a compassionate access programme to enable widespread access. It is also notable that the conclusions of the ACTT paper emphasise the importance of using combination treatment with more than one investigational drug. The RECOVERY has not made any announcement for the use of combination therapy. [12, 13, 14]
An established principal in ethical research, at least from HIV studies, is that no study participant should use less than current standard of care. When the standard of care changes, research studies need to rapidly change too, to ensure their participants do not use anything less.
The RECOVERY researchers have publicised how quickly they launched their study. They also claim that early signals will be acted on quickly to stop ineffective drugs and to prioritise newly effective ones. If this was really true, it shouldn’t have taken a request from the MHRA to look again at the HCQ arm. The investigators and the independent data and safety monitoring board (DSMB) for the study should have done this already. The study should also have publicised whether more recent compounds with positive data have been considered – for example using anticoagulants or ACE inhibitors or the anti-rheumatoid anakinra. [15, 16, 17]
The fact that RECOVERY didn’t stop the HCQ arm based on its own analysis plan, nor announce plans to look at other ongoing arms, are a concern for the study overall, and especially for participants who put their trust in these researchers.
The press release concludes “These data convincingly rule out any meaningful mortality benefit of hydroxychloroquine in patients hospitalised with COVID-19”. If this is really the case, then slightly less convincing data should have been enough to stop this study arm much earlier and allowed participants the option to use other drugs that stood a better chance of benefit.
COMMENT
The degree of failure in this study is not an unfortunate scientific event.
It is a terrible failure.
Many of these issues, including on the data plans and timeline, the decision to continue using HCQ and access to remdesivir were raised by email with the RECOVERY coinvestigator Peter W Horby on 24 May 2020. This email has neither been acknowledged nor answered.
Instead, hundreds of people have died using an intervention that has no signal of benefit, or because they were randomised to standard of care with no potentially active treatment.
These results should prompt an urgent review of the other study arms in the RECOVERY study and an investigation for why such ineffective treatment continued for so long. Even though the study says the DSMB have been reviewing results every two weeks, the predefined rules to close or continue a study might not be appropriate – but as these have been excluded from the protocol it is difficult to comment.
This study – and no doubt others – should be using drugs that have a better indication of efficacy.
Study participants deserve better.
References
- RECOVERY trial statement. Statement from the Chief Investigators of the Randomised Evaluation of COVid-19 thERapY (RECOVERY) Trial on hydroxychloroquine. (5 June 2020).
https://www.recoverytrial.net/files/hcq-recovery-statement-050620-final-002.pdf - Molina JM et al. No evidence of rapid antiviral clearance or clinical benefit with the combination of hydroxychloroquine and azithromycin in patients with severe COVID-19 infection. Médecine et Maladies Infectieuses (2020), doi: 10.1016/j.medmal.2020.03.006.
https://www.sciencedirect.com/science/article/pii/S0399077X20300858 - Geleris J et al. Observational study of hydroxychloroquine in hospitalized patients with Covid-19. DOI: 10.1056/NEJMoa2012410. (7 May 2020).
https://www.nejm.org/doi/full/10.1056/NEJMoa2012410 - MagagnoliJ et al. Outcomes of hydroxychloroquine usage in United States veterans hospitalized with Covid-19. DOI: 10.1101/2020.04.16.20065920. (23 April 2020).
https://www.medrxiv.org/content/10.1101/2020.04.16.20065920v2 - Maisonnasse P et al. Hydroxychloroquine in the treatment and prophylaxis of SARS-CoV-2 infection in non- human primates. Nature Research. In Review. (6 May 2020).
https://www.researchsquare.com/article/rs-27223/v1 - Prescrire. Covid-19 and hydroxychloroquine (Plaquenil): new data show no evidence of efficacy.
https://english.prescrire.org/en/81/168/58639/0/NewsDetails.aspx - RECOVERY trial statement. Recruitment to the RECOVERY trial (including the Hydroxychloroquine arm) REMAINS OPEN. (22 May 2020).
https://www.recoverytrial.net/files/professional-downloads/recovery_noticetoinvestigators_2020-05-24_1422.pdf(PDF) - Lancet retraction: “Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis” (4 June 2020).
https://www.thelancet.com/lancet/article/s0140673620313246 (web)
https://marlin-prod.literatumonline.com/pb-assets/Lancet/pdfs/S0140673620313246.pdf(PDF)
http://i-base.info/htb/37988 (i-Base report) - Collins S. RETRACTED – Re: No benefit from hydroxychloroquine with or without macrolide antibiotics in analysis of 96000 patients. HTB (1 June 2020).
http://i-base.info/htb/37988 - Cao B et al. A trial of lopinavir–ritonavir in adults hospitalized with severe Covid-19. NEJM. DOI: 10.1056/NEJMoa2001282. (18 March 2020).
https://www.nejm.org/doi/full/10.1056/NEJMoa2001282 - Collins S. UK guidelines for the treatment of COVID-19: UK prioritises 42 studies. HTB (1 June 2020).
http://i-base.info/htb/38094 - Beigel JH et al. Remdesivir for the treatment of covid-19 – preliminary report. NEJM. DOI: 10.1056/NEJMoa2007764. (22 May 2020).
https://www.nejm.org/doi/full/10.1056/NEJMoa2007764 - MHRA. MHRA issues a scientific opinion for the first medicine to treat COVID-19 in the UK. (26 May 2020).
https://www.gov.uk/government/news/mhra-supports-the-use-of-remdesivir-as-the-first-medicine-to-treat-covid-19-in-the-uk - MHRA. Early access to medicines scheme (EAMS) scientific opinion: remdesivir in the treatment of patients hospitalised with suspected or laboratory-confirmed SARS-CoV-2 infection who meet the clinical criteria. (26 May 2020).
https://www.gov.uk/government/publications/early-access-to-medicines-scheme-eams-scientific-opinion-remdesivir-in-the-treatment-of-patients-hospitalised-with-suspected-or-laboratory-confirme - Collins S. Anticoagulants associated with improved survival rates in people hospitalised with COVID-19. HTB (14 May 2020).
http://i-base.info/htb/37794 - Mascolini M. ACE inhibitors and angiotensin receptor blockers for hypertension tied to lower death risk with COVID-19. HTB (14 May 2020).
http://i-base.info/htb/37884 - Rheumatoid arthritis drug anakinra in small study to treat COVID-19. HTB (14 May 2020).
http://i-base.info/htb/37863