Statistical Techniques to Explore the Quality of Constraints in Constraint-Based Modeling Environments
- PDF / 906,913 Bytes
- 28 Pages / 439.37 x 666.142 pts Page_size
- 87 Downloads / 143 Views
Statistical Techniques to Explore the Quality of Constraints in Constraint-Based Modeling Environments Jaime Gálvez & Ricardo Conejo & Eduardo Guzmán
Published online: 6 November 2013 # International Artificial Intelligence in Education Society 2013
Abstract One of the most popular student modeling approaches is Constraint-Based Modeling (CBM). It is an efficient approach that can be easily applied inside an Intelligent Tutoring System (ITS). Even with these characteristics, building new ITSs requires carefully designing the domain model to be taught because different sources of errors could affect the efficiency of the system. In this paper a novel mechanism for studying the quality of the elements in the domain model of CBM systems is presented. This mechanism combines CBM with the Item Response Theory (IRT), a data-driven technique for automatic assessment. The goal is to improve the quality of the elements that are used in problem solving environments for assessment or instruction. In this paper we propose a set of statistical techniques, i.e., the analysis of the point-biserial correlation, the Cronbach’s alpha and the information function, to explore the quality of constraints. Two different tools have been used to test this approach: a problem solving environment designed to assess students in project investment analysis; and an independent component that performs assessments using CBM and IRT. Results suggest that the three methods produce consistent diagnosis and may be complementary in some cases. In the experiments we have carried out they were able to detect faulty, bad and good quality constraints. Keywords Problem solving environments . Constraint-based modeling . Item response theory
Introduction Among the existing approaches that can be applied to modeling students in problem solving learning environments, Constraint-Based Modeling (CBM) has proved its J. Gálvez (*) : R. Conejo : E. Guzmán Universidad de Málaga, 29071 Málaga, Spain e-mail: [email protected] R. Conejo e-mail: [email protected] E. Guzmán e-mail: [email protected]
Int J Artif Intell Educ (2013) 23:22–49
23
effectiveness with a range of tutors and studies performed in the last few years (Mitrovic 2012; Mitrovic et al. 2007). It is easier to apply than other approaches, such as Model Tracing (Mitrovic et al. 2003) since CBM does not require the identification of all possible steps a student could take to reach a solution to a problem. On the contrary, only those principles (called constraints) that no solution should violate need to be identified. CBM is an effective paradigm, the power of which lies in the design of the constraints set. This set is the most important element of this modeling paradigm in order to conduct intelligent tutorial actions. Designing the constraints set in new learning environments can be performed very easily using authoring tools such as ASPIRE (Mitrovic et al. 2006) as no programming skills are needed. What is necessary to appropriately model constraints is a broad knowledge of the domain matter; the s
Data Loading...
