History repeating: A roadmap to address common problems in psychedelic science

Despite a decade of optimism about psychedelics for treating mental disorders, the field faces ten pressing challenges that threaten the validity of core findings. These problems undermine internal validity (treatment effects from non-treatment factors), external validity (poor generalizability), construct validity (unclear mechanisms), and statistical conclusion validity (conclusions unsupported by data and methods). The challenges co-occur, strongly limiting conclusions about safety and efficacy. A roadmap and checklist are provided for researchers, journalists, funders, and policymakers to assess study quality. Addressing these issues is necessary to determine whether the optimism is warranted and to avoid repeating past mistakes.

Study at a glance

Abstract

Research in the last decade has expressed considerable optimism about the clinical potential of psychedelics for the treatment of mental disorders. This optimism is reflected in a considerable increase in research papers, investments by pharmaceutical companies, patents, media coverage, as well as political and legislative changes. However, psychedelic science is facing serious challenges that threaten the validity of core findings and raise doubt regarding clinical efficacy and safety. In this paper, we introduce the 10 most pressing challenges, grouped into easy, moderate, and hard problems. We show how these problems threaten internal validity (treatment effects are due to factors unrelated to the treatment), external validity (lack of generalizability), construct validity (unclear working mechanism) or statistical conclusion validity (conclusions do not follow from the data and methods). These problems tend to co-occur in psychedelic studies, strongly limiting conclusions that can be drawn about the safety and efficacy of psychedelic therapy. We provide a roadmap for tackling these challenges in the next decade of psychedelic science, and share a checklist that researchers, journalists, funders, policy makers, and other stakeholders can use to assess the quality of psychedelic science. Addressing today’s problems is necessary to find out whether the optimism regarding the potential of psychedelics has been warranted and to avoid history repeating itself.