Assessing Whether Students Seek Constructive Criticism: The Design of an Automated Feedback System for a Graphic Design
- PDF / 1,277,507 Bytes
- 29 Pages / 439.37 x 666.142 pts Page_size
- 69 Downloads / 174 Views
Assessing Whether Students Seek Constructive Criticism: The Design of an Automated Feedback System for a Graphic Design Task Maria Cutumisu 1 & Kristen P. Blair 2 & Doris B. Chin 2 & Daniel L. Schwartz 2
# International Artificial Intelligence in Education Society 2016
Abstract We introduce a choice-based assessment strategy that measures students’ choices to seek constructive feedback and to revise their work. We present the feedback system of a game we designed to assess whether students choose positive or negative feedback and choose to revise their posters in the context of a poster design task, where they learn graphic design principles from feedback. We then describe an empirical study that sampled one hundred and six students from a US middle school to evaluate the feedback system. We make the following contributions: (1) describe the design and implementation of a novel feedback system embedded in an assessment game, Posterlet, (2) outline an approach to analyze graphic design principles automatically to provide contextual feedback in a novel poster design domain, (3) show that choices to seek negative feedback and to revise correlate with in-game performance, and most importantly, (4) show that choices correlate with in-school achievement: the choice to revise correlated with both in-school performance measures (Science and Mathematics grades), while the choice to seek negative feedback correlated with students’ prior standardized scores in Mathematics. Keywords Choice . Feedback . Performance . Learning . Assessment . Game
* Maria Cutumisu [email protected]
1
Department of Educational Psychology, Faculty of Education, University of Alberta, Edmonton, AB, Canada
2
Graduate School of Education, Stanford University, Stanford, CA, USA
Int J Artif Intell Educ
Introduction The main purpose of formative assessments is to promote learning rather than just Baccountability, ranking, or competence^ (Black and Wiliam 2004, 2009). While the literature shows strong evidence of an increase in students’ performance when formative assessments are employed (Black and Wiliam 1998), these assessments are not always implemented in the classroom. The pervasiveness of massive open online courses (MOOCs) calls for scalable formative assessments that can provide immediate, dynamic, and responsive feedback customized to the user and to the learning domain and context. Most learning environments that support feedback are designed for structured domains, such as mathematics, and provide little agency to the users regarding when and how to receive feedback. Feedback provides information related to a person’s performance or understanding (Hattie and Timperley 2007) and, thus, it is an important, though often neglected, component of the learning process. Consider that most innovations introduced in schools (e.g., computer-assisted instruction, peer tutoring) improve achievement by about 0.4 of a standard deviation (Hattie 1999). In contrast, twelve meta-analyses focusing on feedback in classrooms, which included 196 studie
Data Loading...
