Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System
- PDF / 1,513,430 Bytes
- 19 Pages / 439.37 x 666.142 pts Page_size
- 32 Downloads / 188 Views
Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System Kathryn S. McCarthy 1 & Aaron D. Likens 1 & Amy M. Johnson 1 & Tricia A. Guerrero 1 & Danielle S. McNamara 1
# International Artificial Intelligence in Education Society 2018
Abstract Research suggests that promoting metacognitive awareness can increase performance in, and learning from, intelligent tutoring systems (ITSs). The current work examines the effects of two metacognitive prompts within iSTART, a reading comprehension strategy ITS in which students practice writing quality self-explanations. In addition to comparing iSTART practice to a no-training control, those in the iSTART condition (n = 116) were randomly assigned to a 2 (performance threshold: off, on) × 2(self-assessment: off, on) design. The performance threshold notified students when their average self-explanation score was below an experimenter-set threshold and the self-assessment prompted students to estimate their self-explanation score on the current trial. Students who practiced with iSTART had higher posttest self-explanation scores and inference comprehension scores on a transfer test than students in the no training control, replicating previous benefits for iSTART. However, there were no effects of either metacognitive prompt on these learning outcomes. In-system selfexplanation scores indicated that the metacognitive prompts were detrimental to performance relative to standard iSTART practice. This study did not find benefits of metacognitive prompts in enhancing performance during practice or after the completion of training. Such findings support the idea that improving reading comprehension strategies comes from deliberate practice with actionable feedback rather than explicit metacognitive supports. Keywords Intelligent tutoring systems . Metacognition . Reading comprehension . Log data
* Kathryn S. McCarthy [email protected]
1
Arizona State University, Tempe, AZ, USA
Int J Artif Intell Educ
Introduction Metacognition, or Bthinking about thinking,^ refers to processes related to evaluating what one knows (Flavell 1979). For example, at the close of a challenging class, a student might reflect on how much of the lecture she has actually understood. Later that day, she might make judgments about how long it will take her to review the material and evaluate which specific topics she understands least so that she can spend more time on them. This type of reflection on to-be-learned material is characteristic of skilled metacognition. Metacognition has been shown to relate to a variety of learning outcomes (Hacker et al. 1998; Metcalfe 1996). Unfortunately, students often fail to engage in metacognitive reflection (Pintrich 2000; Varner et al. 2013; Zimmerman 2008; Zimmerman and Schunk 2001) and often inaccurately assess their own learning (Maki 1998). Research in the design of intelligent tutoring systems, or ITSs, has shown that the inclusion of metacognitive prompts can increase performance in the system as well a
Data Loading...
