NotesFAQContact Us
Search Tips
ERIC Number: ED560513
Record Type: Non-Journal
Publication Date: 2015-Jun
Pages: 8
Abstractor: As Provided
Reference Count: 19
Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement
Matsuda, Noboru; Furukawa, Tadanobu; Bier, Norman; Faloutsos, Christos
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (8th, Madrid, Spain, Jun 26-29, 2015)
How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester's worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have developed an innovative method to discover skill models from the data of online courses. Our method assumes that online courses have a pre-defined skill map for which skills are associated with formative assessment items embedded throughout the online course. Our method carefully exploits correlations between various parts of student performance, as well as in the text of assessment items, to build a superior statistical model that even outperforms human experts. To evaluate our method, we compare our method with existing methods (LFA) and human engineered skill models on three Open Learning Initiative (OLI) courses at Carnegie Mellon University. The results show that (1) our method outperforms human-engineered skill models, (2) skill models discovered by our method are interpretable, and (3) our method is remarkably faster than existing methods. These results suggest that our method provides a significant contribution to the evidence-based, iterative refinement of online courses with a promising scalability. [For complete proceedings, see ED560503.]
International Educational Data Mining Society. e-mail:; Web site:
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: International Educational Data Mining Society
Identifiers - Location: Pennsylvania
IES Grant or Contract Numbers: 1418244