A computational model for subjective evaluation of novelty in descriptive aptitude
- PDF / 2,527,472 Bytes
- 38 Pages / 439.37 x 666.142 pts Page_size
- 82 Downloads / 199 Views
A computational model for subjective evaluation of novelty in descriptive aptitude Nandita Bhanja Chaudhuri1 · Debayan Dhar1 · Pradeep G. Yammiyavar1 Accepted: 4 November 2020 © Springer Nature B.V. 2020
Abstract Evaluating novelty in design education is subjective and generally depends on expert’s referential metrics. Presently, practitioners in this field perform subjective evaluation of answers of prospective students, but many a time, humans are prone to errors when associated with repetitive tasks on large-scale. Therefore, this paper attempts to automate the process of evaluating novelty by a proposed computational model. The present study explores design aptitude to evaluate novelty in solutions provided by students in an examination. Mixed-methods research is conducted based on structured questionnaire and analysis to investigate features of subjective evaluation of novelty practiced for evaluation in design education. The survey resulted in features that closely resemble human evaluation strategies for evaluating novelty from descriptive solutions. Further, a computational model is proposed, designed, and implemented that evaluates novelty. Scores are generated for each feature by unsupervised learning techniques, eventually calculating novelty score by a scoring function. This model suggests unambiguous scores to solutions, which might help in a consistent selection of students aspiring admission to design schools. This study attempts to reduce pain points of educational practitioners by offering a voluntary automated technique for subjective evaluation and optimize trustworthiness of students in examination process. In future, this model can be extended for evaluating any other domain of interest. Keywords Design education · Examination · Assessment · Unsupervised learning · Machine learning techniques
* Debayan Dhar [email protected] Nandita Bhanja Chaudhuri [email protected] Pradeep G. Yammiyavar [email protected] 1
Department of Design, Indian Institute of Technology Guwahati, Guwahati, Assam, India
13
Vol.:(0123456789)
N. B. Chaudhuri et al.
Introduction Education is a complex process with enormous pedagogic activities. The examination system being part of it is a significant process that determines the performance of students. Evaluation procedure of any examination system mainly comprises of two categories: (1) Objective evaluation and (2) Subjective evaluation (Thomas et al. 2008). Objective evaluation is associated with questions that have specific answers. Presently, majority of exams that are objective in nature are automated and requires mapping of questions with their corresponding answers. Subjective evaluation on the other hand is highly complex as it comprises of descriptive answers that are evaluated based on examiners choice, knowledge, and belief, specifically in evaluating solutions related to design and creativity where evaluation of novelty is essential. Evaluating such category of answers might be challenging as it is necessary to maintain consistency of
Data Loading...
