From efficacy to effectiveness: evaluating psychedelic randomised controlled trials for trustworthy evidence-based policy and practice.

Regulatory approval of MDMA-assisted therapy for PTSD by the FDA faces epistemological and methodological challenges, particularly the demand for two successful phase 3 randomized controlled trials when there is no agreement on what constitutes success for psychoactive drugs combined with therapy. These complex treatment arrangements undermine the internal validity of estimated average treatment effects compared to conventional controls. The paper reviews assumptions behind RCTs' gold-standard status in evidence-based medicine, emphasizing the need to avoid the extrapolation fallacy. Trustworthiness that efficacy in RCTs will predict effectiveness in target populations depends on the type of psychedelic treatment regulated: low for stand-alone drugs, high for drug-assisted psychotherapies, because these involve different causal claims with distinct external validities.

Study at a glance

Abstract

The recent review of a new drug application for MDMA-assisted therapy for post-traumatic stress disorder by the United States’ Food and Drug Administration (FDA) highlighted epistemological and methodological challenges for evidence assessments. Similar challenges will also be faced in reviews of other compounds in early- and late-stage development, like psilocybin for depression. The regulatory demand for two successful phase 3 randomised controlled trials (RCTs) seems problematic, given a current lack of agreement on what constitutes “success”, particularly when psychoactive drug administration is concomitant with (psycho)therapy. These complex arrangements challenge the internal validity of estimated average treatment effect through comparison with conventional control conditions. This paper reviews the assumptions behind RCTs’ current “gold-standard” status in the hierarchy of evidence-based medicine (EBM). Recapitulating known epistemic limits of randomisation and blinding, it emphasises the urgent need to avoid the extrapolation fallacy. The resulting argument is that the degree of trustworthiness that efficacy — reported in RCTs — will reliably predict effectiveness — in target populations outside RCTs — depends on what type of psychedelic treatments will be regulated. If “stand-alone” drugs for large scale prescription and consumption, trustworthiness should be graded low. On the other hand, for regulation of drug-assisted (psycho)therapies, the degree of trustworthiness can be considered high. The reason being that these two treatment approaches are based on different causal claims with distinct external validities. Therefore, careful assessment of support factors in each is recommended to prevent detrimental consequences, from potential rejection of effective therapies up to medical reversal of eventually approved drugs.