The reliability characteristics of the REFLECT rubric for assessing reflective capacity through expressive writing assig
- PDF / 278,428 Bytes
- 5 Pages / 595 x 842 pts (A4) Page_size
- 87 Downloads / 153 Views
Perspect Med Educ https://doi.org/10.1007/s40037-020-00611-2
The reliability characteristics of the REFLECT rubric for assessing reflective capacity through expressive writing assignments: A replication study Lawrence Grierson Joyce Zazulak
· Samantha Winemaker · Alan Taniguchi · Michelle Howard · Denise Marshall ·
© The Author(s) 2020
Abstract Introduction The medical education community has implemented writing exercises that foster critical analysis and nurture reflective capacity. The REFLECT rubric (Wald et al. 2012) was developed to address the challenge of assessing these written reflections. The objective of this replication work is to explore the reproducibility of the reliability characteristics presented by the REFLECT developers. Methods Five raters evaluated narratives written by medical students and experienced clinicians using the REFLECT rubric. Reliability across rubric domains was determined via intraclass correlation coefficient and internal consistency was determined via Cronbach’s alpha. Results Intraclass coefficients demonstrated poor reliability for ratings across all tool criteria (0.350–0.452) including overall ratings of narratives (0.448). Moreover, the internal consistency between scale items was also poor across all criteria (0.529–0.621). Discussion We did not replicate the reliability characteristics presented in the original REFLECT article. We consider these findings with respect to the contextual differences that existed between our study and the Wald and colleagues study, pointing particularly at the possible influence that repetitive testing and refinement of the tool may have had on their reviewers’ shared understanding of its use. We conclude with L. Grierson () · S. Winemaker · A. Taniguchi · M. Howard · D. Marshall · J. Zazulak Department of Family Medicine, McMaster University, Hamilton, ON, Canada [email protected] L. Grierson McMaster Education Research, Innovation, and Theory (MERIT), Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
a discussion about the challenges inherent to reductionist approaches to assessing reflection. Keywords Reflection · Assessment · Reliability · Summative · Formative
Introduction Reflection is a metacognitive activity that involves thinking intentionally about performance before, during, or after situations with the aim of detecting and characterizing the mental models that underpin the decisions and actions relevant to the performance outcome [1, 2]. These models refer to the representations that one has of the relationships that exist between various aspects of the world in which they perform. When healthcare practitioners reflect, it allows them to perceive information from clinical encounters in a way that has the potential to inform their practice in future encounters. As such, it is seen as an essential habit to nurture in new physicians. Accordingly, many medical training programs have adopted writing exercises as a way to develop the capabilities of reflection. In these and other similar assignments, lear
Data Loading...
