Utilizing Game Analytics to Inform and Validate Digital Game-based Assessment with Evidence-centered Game Design: A Case
- PDF / 839,434 Bytes
- 23 Pages / 439.37 x 666.142 pts Page_size
- 26 Downloads / 195 Views
Utilizing Game Analytics to Inform and Validate Digital Game-based Assessment with Evidence-centered Game Design: A Case Study Fu Chen 1 & Ying Cui 1 & Man-Wai Chu 2 Received: 13 January 2020 / Revised: 6 May 2020 / Accepted: 28 June 2020 # International Artificial Intelligence in Education Society 2020
Abstract The purpose of this case study is to demonstrate how to utilize machine learning approaches to analyze student process data for validating and informing digital gamebased assessments (DGBAs) with an evidence-centered game design (ECgD). The first analysis was conducted to examine whether students’ mastery of the overall skill required by the game can be well predicted by task-related behavioral features and if the selected key features map onto the evidence model of the ECgD. Specifically, we extracted 27 behavioral features as the indicators of students’ gameplay activities from the evidence trace files and modelled them using a machine learning algorithm— support vector machine with recursive feature elimination—to identify the key features for prediction. The key features were in turn used to predict students’ mastery of the overall skill. Results showed that students’ retry attempts on two assessment tasks were found to be most influential for prediction with a moderate to high training and testing accuracy. The second analysis was conducted to examine whether the number of learning opportunities is sufficient for evaluating students’ mastery of the overall skill as well as determine the optimal number of learning opportunities for evaluation. The approach of long short-term memory networks was used to model students’ time-series behavioral features across multiple learning opportunities for predicting their acquisition of the overall skill. Results suggested that five learning opportunities were a good balance between evaluation accuracy and practical feasibility, and they were sufficient for evaluating students’ mastery of the overall skill given the DGGA tasks. Keywords Game analytics . Digital game-based assessment . Evidence-centered game
design . Evidence trace files . Machine learning
* Fu Chen [email protected] Extended author information available on the last page of the article
International Journal of Artificial Intelligence in Education
Introduction Digital game-based learning has received increasing interest from various educational domains in recent years (Hwang and Wu 2012) such as language (Yukselturk et al. 2018), history (Kazanidis et al. 2018), computer science (Mathrani et al. 2016), mathematics (Kiili and Ketamo 2017) and geography (Gaydos 2016). The popularity of digital gamebased learning stems from the shifting workplace requirements in the digital age with an increasing emphasis of 21st century skills (e.g., Organization for Economic Co-operation and Development 2005) for future career success (Griffin et al. 2012). However, traditional student learning in schools still heavily rely on teacher-centred, lecture-oriented and authoritarian instruction which may not adequately a
Data Loading...
