Is mindfulness research methodology improving over time? A systematic review

A systematic review of 142 randomized clinical trials of mindfulness-based interventions for clinical disorders, published between 2000 and 2016, found no statistically significant increases in six key methodological quality indicators over that period, though changes generally trended toward improvement. When analyses were restricted to studies from Europe and North America, a significant increase in reporting of intent-to-treat analyses emerged. Excluding one early high-quality study revealed improvements in sample size, treatment fidelity assessment, and intent-to-treat reporting. Overall, the findings indicate only modest adoption of recommended methodological improvements across the field.

Study at a glance

Abstract

BACKGROUND: Despite an exponential growth in research on mindfulness-based interventions, the body of scientific evidence supporting these treatments has been criticized for being of poor methodological quality. OBJECTIVES: The current systematic review examined the extent to which mindfulness research demonstrated increased rigor over the past 16 years regarding six methodological features that have been highlighted as areas for improvement. These feature included using active control conditions, larger sample sizes, longer follow-up assessment, treatment fidelity assessment, and reporting of instructor training and intent-to-treat (ITT) analyses. DATA SOURCES: We searched PubMed, PsychInfo, Scopus, and Web of Science in addition to a publically available repository of mindfulness studies. STUDY ELIGIBILITY CRITERIA: Randomized clinical trials of mindfulness-based interventions for samples with a clinical disorder or elevated symptoms of a clinical disorder listed on the American Psychological Association's list of disorders with recognized evidence-based treatment. STUDY APPRAISAL AND SYNTHESIS METHODS: Independent raters screened 9,067 titles and abstracts, with 303 full text reviews. Of these, 171 were included, representing 142 non-overlapping samples. RESULTS: Across the 142 studies published between 2000 and 2016, there was no evidence for increases in any study quality indicator, although changes were generally in the direction of improved quality. When restricting the sample to those conducted in Europe and North America (continents with the longest history of scientific research in this area), an increase in reporting of ITT analyses was found. When excluding an early, high-quality study, improvements were seen in sample size, treatment fidelity assessment, and reporting of ITT analyses. CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS: Taken together, the findings suggest modest adoption of the recommendations for methodological improvement voiced repeatedly in the literature. Possible explanations for this and implications for interpreting this body of research and conducting future studies are discussed.