History repeating: guidelines to address common problems in psychedelic science

Despite a decade of optimism about psychedelics for treating mental disorders, psychedelic science faces serious challenges that threaten the validity of core findings. The paper identifies 10 pressing problems grouped into easy, moderate, and hard categories, showing how they undermine internal validity (treatment effects due to unrelated factors), external validity (lack of generalizability), construct validity (unclear working mechanisms), and statistical conclusion validity (conclusions not supported by data). These problems often co-occur, limiting conclusions about safety and efficacy. The authors provide a roadmap and checklist for stakeholders to assess study quality, emphasizing that addressing these issues is necessary to determine whether therapeutic optimism is warranted and to avoid repeating past mistakes.

Study at a glance

Abstract

Research in the last decade has expressed considerable optimism about the clinical potential of psychedelics for the treatment of mental disorders. This optimism is reflected in an increase in research papers, investments by pharmaceutical companies, patents, media coverage, as well as political and legislative changes. However, psychedelic science is facing serious challenges that threaten the validity of core findings and raise doubt regarding clinical efficacy and safety. In this paper, we introduce the 10 most pressing challenges, grouped into easy, moderate, and hard problems. We show how these problems threaten internal validity (treatment effects are due to factors unrelated to the treatment), external validity (lack of generalizability), construct validity (unclear working mechanism), or statistical conclusion validity (conclusions do not follow from the data and methods). These problems tend to co-occur in psychedelic studies, limiting conclusions that can be drawn about the safety and efficacy of psychedelic therapy. We provide a roadmap for tackling these challenges and share a checklist that researchers, journalists, funders, policymakers, and other stakeholders can use to assess the quality of psychedelic science. Addressing today’s problems is necessary to find out whether the optimism regarding the therapeutic potential of psychedelics has been warranted and to avoid history repeating itself.