A Multimodal Analysis of Making
- PDF / 1,766,584 Bytes
- 35 Pages / 439.37 x 666.142 pts Page_size
- 104 Downloads / 176 Views
A Multimodal Analysis of Making Marcelo Worsley 1
& Paulo
Blikstein 2
# International Artificial Intelligence in Education Society 2017
Abstract This paper presents three multimodal learning analytic approaches from a hands-on learning activity. We use video, audio, gesture and bio-physiology data from a two-condition study (N = 20), to identify correlations between the multimodal data, experimental condition, and two learning outcomes: design quality and learning. The three approaches incorporate: 1) human-annotated coding of video data, 2) automated coding of gesture, audio and bio-physiological data and, 3) concatenated humanannotated and automatically annotated data. Within each analysis we employ the same machine learning and sequence mining techniques. Ultimately we find that each approach provides different affordances depending on the similarity metric and the dependent variable. For example, the analysis based on human-annotated data found strong correlations among multimodal behaviors, experimental condition, success and learning, when we relaxed constraints on temporal similarity. The second approach performed well when comparing students’ multimodal behaviors as a time series, but was less effective using the temporally relaxed similarity metric. The take-away is that there are several strategies for doing multimodal learning analytics, and that many of these approaches can provide a meaningful glimpse into a complex data set, glimpses that may be difficult to identify using traditional approaches. Keywords Learning analytics . Signal processing . Constructionism
Introduction The twenty-first century has seen an expansion in the set of tools available for assessing the quality of a given learning environment (Baker and Yacef 2009; Blikstein and Worsley 2016; Martin and Sherin 2013). A number of the traditional tools: test and quiz
* Marcelo Worsley [email protected]
1
Northwestern University, Evanston, IL, USA
2
Stanford University, Stanford, CA, USA
Int J Artif Intell Educ
performance, speeches and essays; are modes of expression that have been around for centuries and remain the more privileged forms of assessment. For all of their pedagogical shortcomings, these forms of assessment have the benefit of being widely accepted and easy to interpret. However, contemporary learning sciences research is increasingly concerned with additional constructs: motivation, engagement, collaboration, creativity, critical thinking, and problem solving, for example. These are constructs that tend to be much harder to quantify using traditional testing instruments and often necessitate adopting an alternative approach that more closely aligns with the design of constructivist-inspired learning environments (Piaget 1973; Schwartz et al. 2009). By virtue of the breadth of interactions students have with collaborators and various technological resources, traditional tools and metrics are probably not well suited for making 1 or other constructionist-based learning environments. Instead, studyin
Data Loading...
