NotesFAQContact Us
Search Tips
ERIC Number: ED537219
Record Type: Non-Journal
Publication Date: 2012-Jun
Pages: 4
Abstractor: As Provided
Reference Count: 10
Calculating Probabilistic Distance to Solution in a Complex Problem Solving Domain
Sudol, Leigh Ann; Rivers, Kelly; Harris, Thomas K.
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (5th, Chania, Greece, Jun 19-21, 2012)
In complex problem solving domains, correct solutions are often comprised of a combination of individual components. Students usually go through several attempts, each attempt reflecting an individual solution state that can be observed during practice. Classic metrics to measure student performance over time rely on counting the number of submissions or focusing on time taken to complete the problem correctly. These metrics are not robust to the correction of errors that may increase problem solving time, and do not reflect topical misunderstandings on the part of the student. In this paper we propose a metric to measure the probabilistic distance between an observed student solution and a correct solution. Students working in an online programming environment completed four practice problems. Their submissions were then evaluated against a model of the algorithmic components necessary for a correct solution. A Markov Model was used to generate a problem state graph. Our proposed Probabilistic Distance to Solution (PDS) metric was applied to the graph to determine the distance, in program states, from an observed program model to the model of a correct solution. Results indicate that the PDS is useful in determining if an edit or student path is (a) typical of students who have mastered content, and (b) productive in progressing toward a solution. We offer implementation details of PDS and implications for future work based upon current observations. (Contains 2 figures, 1 table, and 3 footnotes.)[For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (5th, Chania, Greece, June 19-21, 2012)," see ED537074.]
International Educational Data Mining Society. e-mail:; Web site:
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: International Educational Data Mining Society