How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex
- PDF / 2,457,912 Bytes
- 17 Pages / 595.276 x 790.866 pts Page_size
- 52 Downloads / 188 Views
How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex Student Construct Shannon H. Sung 1
&
Chenglu Li 2 & Guanhua Chen 3 & Xudong Huang 3 & Charles Xie 1 & Joyce Massicotte 3 & Ji Shen 4
# Springer Nature B.V. 2020
Abstract In this paper, we demonstrate how machine learning could be used to quickly assess a student’s multimodal representational thinking. Multimodal representational thinking is the complex construct that encodes how students form conceptual, perceptual, graphical, or mathematical symbols in their mind. The augmented reality (AR) technology is adopted to diversify student’s representations. The AR technology utilized a low-cost, high-resolution thermal camera attached to a smartphone which allows students to explore the unseen world of thermodynamics. Ninth-grade students (N = 314) engaged in a prediction–observation– explanation (POE) inquiry cycle scaffolded to leverage the augmented observation provided by the aforementioned device. The objective is to investigate how machine learning could expedite the automated assessment of multimodal representational thinking of heat energy. Two automated text classification methods were adopted to decode different mental representations students used to explain their haptic perception, thermal imaging, and graph data collected in the lab. Since current automated assessment in science education rarely considers multilabel classification, we resorted to the help of the state-of-the-art deep learning technique—bidirectional encoder representations from transformers (BERT). The BERT model classified open-ended responses into appropriate categories with higher precision than the traditional machine learning method. The satisfactory accuracy of deep learning in assigning multiple labels is revolutionary in processing qualitative data. The complex student construct, such as multimodal representational thinking, is rarely mutually exclusive. The study avails a convenient technique to analyze qualitative data that does not satisfy the mutual-exclusiveness assumption. Implications and future studies are discussed. Keywords Heat transfer . Representational thinking . Augmented observation . Bidirectional encoder representations from transformers (BERT) . Transfer learning . Automated text classification
Introduction External representations display abstract concepts with concrete symbols or analogy (Shen and Confrey 2007; Xie 2011; Namdar and Shen 2015). Representational thinking, which
* Shannon H. Sung [email protected] 1
Institute for Future Intelligence, 26 Rockland St., Natick, MA 01760, USA
2
School of Teaching and Learning, University of Florida, 1221 SW 5th Ave, Gainesville, FL 32601, USA
3
Concord Consortium, 25 Love Lane, Concord, MA 01742, USA
4
Department of Teaching and Learning, University of Miami, 5202 University Drive, Coral Gables, FL 33124, USA
could be operationally defined as the construct encoding how one mentally forms conceptual, perceptual, graphical, or mathe
Data Loading...
