Applying Natural Language Processing and Hierarchical Machine Learning Approaches to Text Difficulty Classification
- PDF / 899,408 Bytes
- 34 Pages / 439.37 x 666.142 pts Page_size
- 20 Downloads / 254 Views
Applying Natural Language Processing and Hierarchical Machine Learning Approaches to Text Difficulty Classification Renu Balyan 1
& Kathryn
S. McCarthy 2 & Danielle S. McNamara 3
# International Artificial Intelligence in Education Society 2020
Abstract For decades, educators have relied on readability metrics that tend to oversimplify dimensions of text difficulty. This study examines the potential of applying advanced artificial intelligence methods to the educational problem of assessing text difficulty. The combination of hierarchical machine learning and natural language processing (NLP) is leveraged to predict the difficulty of practice texts used in a reading comprehension intelligent tutoring system, iSTART. Human raters estimated the text difficulty level of 262 texts across two text sets (Set A and Set B) in the iSTART library. NLP tools were used to identify linguistic features predictive of text difficulty and these indices were submitted to both flat and hierarchical machine learning algorithms. Results indicated that including NLP indices and machine learning increased accuracy by more than 10% as compared to classic readability metrics (e.g., Flesch-Kincaid Grade Level). Further, hierarchical outperformed non-hierarchical (flat) machine learning classification for Set B (72%) and the combined set A + B (65%), whereas the nonhierarchical approach performed slightly better than the hierarchical approach for Set A (79%). These findings demonstrate the importance of considering deeper features of language related to text difficulty as well as the potential utility of hierarchical machine learning approaches in the development of meaningful text difficulty classification. Keywords Text difficulty . Machine learning . Hierarchical classification . Natural
language processing
* Renu Balyan [email protected] Kathryn S. McCarthy [email protected] Danielle S. McNamara [email protected] Extended author information available on the last page of the article
International Journal of Artificial Intelligence in Education
Introduction Text remains a crucial learning tool in most classrooms today (Fuchs et al. 2014). In the classroom, students are often tasked to read texts and textbooks in order to learn new information. As such, many educators and text publishers attempt to level texts such that text difficulty is appropriate for the students (National Governors Association Center for Best Practices 2010). Readability formulas have been used for well over a century as a means to evaluate text difficulty. Indeed, teachers have long relied on readability metrics to select classroom materials (e.g., Fry 2002; Chall 1988). In many ways, this practice is well grounded: theories of learning suggest that learning occurs most readily when tasks are tailored to students’ ability (e.g., Bjork 1994; Vygotsky 1978). Vygotsky’s well-known zone of proximal development posits that tasks that are challenging, but potentially achievable with adequate support, are more effective for learning than tasks that are too easy o
Data Loading...
