Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Interventio
- PDF / 387,958 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 43 Downloads / 173 Views
Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Intervention Kendal Holtrop 1
&
Debra L. Miller 1 & Jared A. Durtschi 2 & Marion S. Forgatch 3,4
Accepted: 12 October 2020 # The Author(s) 2020
Abstract Empirically determining the components of evidence-based interventions contributing to positive change is a crucial, yet understudied area of research. In support of this aim, we describe the development and evaluation of an observational rating system for measuring fidelity to specific components of the evidence-based GenerationPMTO parenting intervention. A five-step process was employed to systematically develop the rating system, which included consultation with the intervention developer and input from additional GenerationPMTO experts. The rating system was then tested using 247 h of video data from 184 parenting group intervention sessions. Study findings support the psychometric properties of the new measure with regard to item performance, reliability (i.e., inter-rater reliability of items, dimensionality of components, internal consistency of component scales), and validity (i.e., content validity, convergent validity, discriminant validity, and predictive validity of the component scales) for seven of the eight scales evaluated. The seven components include clear directions, skill encouragement, emotion regulation, limit setting, effective communication, problem solving, and monitoring. Data did not support the psychometric properties of the positive involvement scale. Overall, the ability to assess component-specific fidelity allows for a more nuanced examination of change processes, with meaningful implications for research and practice. Keywords Active ingredients . Component level fidelity . Component level implementation fidelity rating system . Component-specific fidelity . Fidelity . GenerationPMTO
Empirically determining the components of evidence-based interventions responsible for achieving positive change is a crucial, yet understudied area of research (Abry et al. 2015; Blase and Fixsen 2013; Weisz and Kazdin 2017). The intervention testing process is too often limited to comparing average change between randomly assigned conditions; yet, the change process itself remains shrouded in a figurative “black box” (Abry et al. 2015; Nelson et al. 2012). This lack of Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11121-020-01177-5) contains supplementary material, which is available to authorized users. * Kendal Holtrop [email protected] 1
Department of Human Development and Family Studies, Michigan State University, 552 W. Circle Drive, East Lansing, MI 48824, USA
2
School of Family Studies and Human Services, Kansas State University, Manhattan, KS, USA
3
Implementation Sciences International, Inc., Eugene, OR, USA
4
Oregon Social Learning Center, Eugene, OR, USA
understanding regarding how and why evidence-based interventions achieve positive change is a critical barrier impeding
Data Loading...