A Validation Study of the Mindfulness-Based Interventions Teaching Assessment Criteria for Assessing Mindfulness-Based Intervention Teacher Skill: Inter-Rater Reliability and Predictive Validity.
Global advances in integrative medicine and health – January 01, 2024
Source: PubMed
Summary
Mindfulness-based interventions (MBIs) show promise in reducing anxiety, with higher teaching quality linked to better outcomes. In a study of 21 MBSR teachers rated by three independent assessors, inter-rater reliability improved significantly with multiple raters, yielding ICCs from 0.6 to 0.8. Among 152 MBSR students, anxiety, depression, and fatigue showed notable improvements over four months, with reductions of 2.3 to 6.3 points on PROMIS measures. Notably, each unit increase in MBI:TAC ratings correlated with a 0.31 decrease in anxiety scores, highlighting the importance of intervention fidelity in mindfulness training.
Abstract
Prior data suggests the Mindfulness-Based Interventions: (MBI) Teaching Assessment Criteria (MBI:TAC) has good inter-rater reliability, but many raters knew teacher experience level. We sought to further evaluate the MBI-TAC's inter-rater reliability and obtain preliminary data on predictive validity. We videorecorded 21 MBSR teachers from academic and community settings. We trained 19 experienced MBI teachers in using the MBI:TAC. MBSR teachers were rated by three assessors; teachers and their assessors did not know one another. To assess predictive validity, MBSR students in courses taught by 18 of the MBSR teachers were invited to complete PROMIS-29 measures before the MBSR course, at the end of the course (month 2), and month 4. Intraclass correlation coefficients (ICCs) representing a single rater ranged from 0.33 to 0.56 on the 6 MBI:TAC domains. Using an average of two raters, ICC estimates ranged from 0.48 to 0.71 and ICCs generalizing to an average of three raters ranged from 0.6 to 0.8. Among n = 152 participating MBSR students, we found improvements from baseline to 2 months and 4 months in PROMIS measures of Anxiety, Depression, Fatigue, Sleep, and Social Role function (range in improvement 2.3 to 6.3, P < 0.0001 for all comparisons except Social Role at 2 months, P = 0.007). Higher MBI:TAC ratings were associated with greater improvements in anxiety among MBSR students from baseline to 2 months, with a -0.31 lower participant anxiety score per 1 unit increase in MBI:TAC composite teaching rating (95% CI -0.58, -0.05, P = 0.019), but we did not find statistically significant relationships with improvements in other PROMIS-29 domains. ICCs indicated good reliability using an average of three ratings, but inter-rater reliability was only fair using a single rater. We found initial validation that higher MBI:TAC ratings predicted greater improvements in anxiety symptoms in MBSR participants.