ERIC Number: ED560517
Record Type: Non-Journal
Publication Date: 2015-Jun
Pages: 8
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
A Framework for Multifaceted Evaluation of Student Models
Huang, Yun; González-Brenes, José P.; Kumar, Rohit; Brusilovsky, Peter
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (8th, Madrid, Spain, Jun 26-29, 2015)
Latent variable models, such as the popular Knowledge Tracing method, are often used to enable adaptive tutoring systems to personalize education. However, finding optimal model parameters is usually a difficult non-convex optimization problem when considering latent variable models. Prior work has reported that latent variable models obtained from educational data vary in their predictive performance, plausibility, and consistency. Unfortunately, there are still no unified quantitative measurements of these properties. This paper suggests a general unified framework (that we call Polygon) for multifaceted evaluation of student models. The framework takes all three dimensions mentioned above into consideration and offers novel metrics for the quantitative comparison of different student models. These properties affect the effectiveness of the tutoring experience in a way that traditional predictive performance metrics fall short. The present work demonstrates our methodology of comparing Knowledge Tracing with a recent model called Feature-Aware Student Knowledge Tracing (FAST) on datasets from different tutoring systems. Our analysis suggests that FAST generally improves on Knowledge Tracing along all dimensions studied. [For complete proceedings, see ED560503.]
Descriptors: Guidelines, Models, Prediction, Evaluation Methods, Statistical Analysis, Comparative Analysis, Intelligent Tutoring Systems, Instructional Effectiveness, Physics, Probability, Geometry, Computer Software, Mathematics Instruction, Science Instruction, Online Courses, Data Analysis
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Office of Naval Research (ONR)
Authoring Institution: International Educational Data Mining Society
Grant or Contract Numbers: N00014-12-C-0535