Ecological Momentary Assessment as a Measure of Intervention Change: Evaluation in 4 Digital Mental Health Trials
Christian A. Webb, Lori M. Hilt, Caroline M. Swords, Daniel M. Bolt, Hadar Fisher, Simon B. Goldberg
October 17, 2023 preprint DOI: 10.31234/osf.io/3xvck via OpenAlex
Summary
Ecological momentary assessment (EMA) measures of rumination are only modestly correlated with conventional self-report measures, especially for change over time, partly due to lower reliability of EMA. Changes in rumination were larger for conventional self-report than EMA. Both types of measures accounted for unique variance in depressive symptom improvement, showing incremental predictive validity. The findings suggest that EMA and conventional self-report provide distinct, clinically meaningful information. Researchers using EMA should consider psychometric properties and the precise construct they intend to capture.
Study at a glance
| Characteristics | Secondary analysis of data from 4 app-based meditation trials |
|---|---|
| Sample size | 412 |
| Population | Participants in 4 app-based meditation trials |
| Keywords | Rumination Intervention counseling Point geometry Ecology Psychology |
| Citations | 1 |
| Key finding | Conventional self-report and EMA measures of rumination provide distinct and clinically meaningful information, each accounting for unique variance in depressive symptom improvement. |
Abstract
Ecological momentary assessment (EMA) is increasingly being incorporated into intervention studies to acquire a more fine-grained and ecologically valid assessment of change. The added utility of including relatively burdensome EMA measures in a clinical trial hinges on several psychometric assumptions, including that these measure are: (1) reliable, (2) related but not redundant with conventional self-report measures (convergent and discriminant validity), (3) sensitive to intervention-related change, and (4) associated with a clinically-relevant criterion of improvement (criterion validity) above conventional self-report measures (incremental validity). Using data from 4 app-based meditation trials (N = 412), we examined the reliability, validity, and sensitivity to change of conventional self-report and EMA measures of improvement in rumination. Conventional self-report and EMA measures of rumination were only modestly correlated, particularly with regards to change over time, which may be due to their lower reliability. Changes in rumination were larger for conventional self-report than EMA. Notably, change in both self-report and EMA rumination each accounted for unique variance in depressive symptom improvement, demonstrating incremental predictive validity. Conventional self-report and EMA measures of rumination provide distinct and clinically meaningful information. Researchers using EMA should consider the psychometric properties of their measures and the precise construct they intend to capture.