A Validation Study of the Mindfulness-Based Interventions Teaching Assessment Criteria for Assessing Mindfulness-Based Intervention Teacher Skill: Inter-Rater Reliability and Predictive Validity.

Teaching quality in mindfulness-based stress reduction (MBSR) courses, measured by the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC), shows fair to good inter-rater reliability depending on the number of raters. Using a single rater, reliability was fair (intraclass correlation coefficients 0.33–0.56 across six domains); with three raters, reliability was good (0.6–0.8). Among 152 MBSR students, anxiety, depression, fatigue, sleep, and social role function improved from before the course to two and four months later (improvements of 2.3 to 6.3 points). Higher MBI:TAC ratings predicted greater anxiety reduction: each one-unit increase in composite teaching rating was associated with a 0.31-point greater decrease in anxiety score. No significant relationships were found for other health domains.

Study at a glance

Abstract

Prior data suggests the Mindfulness-Based Interventions: (MBI) Teaching Assessment Criteria (MBI:TAC) has good inter-rater reliability, but many raters knew teacher experience level. We sought to further evaluate the MBI-TAC's inter-rater reliability and obtain preliminary data on predictive validity. We videorecorded 21 MBSR teachers from academic and community settings. We trained 19 experienced MBI teachers in using the MBI:TAC. MBSR teachers were rated by three assessors; teachers and their assessors did not know one another. To assess predictive validity, MBSR students in courses taught by 18 of the MBSR teachers were invited to complete PROMIS-29 measures before the MBSR course, at the end of the course (month 2), and month 4. Intraclass correlation coefficients (ICCs) representing a single rater ranged from 0.33 to 0.56 on the 6 MBI:TAC domains. Using an average of two raters, ICC estimates ranged from 0.48 to 0.71 and ICCs generalizing to an average of three raters ranged from 0.6 to 0.8. Among n = 152 participating MBSR students, we found improvements from baseline to 2 months and 4 months in PROMIS measures of Anxiety, Depression, Fatigue, Sleep, and Social Role function (range in improvement 2.3 to 6.3, P < 0.0001 for all comparisons except Social Role at 2 months, P = 0.007). Higher MBI:TAC ratings were associated with greater improvements in anxiety among MBSR students from baseline to 2 months, with a -0.31 lower participant anxiety score per 1 unit increase in MBI:TAC composite teaching rating (95% CI -0.58, -0.05, P = 0.019), but we did not find statistically significant relationships with improvements in other PROMIS-29 domains. ICCs indicated good reliability using an average of three ratings, but inter-rater reliability was only fair using a single rater. We found initial validation that higher MBI:TAC ratings predicted greater improvements in anxiety symptoms in MBSR participants.