NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1040135
Record Type: Journal
Publication Date: 2014
Pages: N/A
Abstractor: As Provided
Reference Count: N/A
ISSN: ISSN-0161-4681
Toward the Relational Management of Educational Measurement Data
Chung, Gregory K. W. K.
Teachers College Record, v116 n11 2014
Background: Historically, significant advances in scientific understanding have followed advances in measurement and observation. As the resolving power of an instrument increased, so have gains in the understanding of the phenomena being observed. Modern interactive systems are potentially the new "microscopes" when they are instrumented to record fine-grained observations of what students do in an online task. Advances in the conceptualization, design, and analyses of such interaction data enable the discovery of learning patterns and can power new applications. One application, personalization, is one of 14 engineering Grand Challenges identified by the National Academy of Engineering (2008). Purpose: This article examines three levels of data available in online systems that can be used to understand student performance. Empirical research is reviewed to examine three fundamental questions: To what extent does students' online behavior (a) relate to their cognitive processing, and to what extent can student behavior (b) be used to model their problem solving process and (c) be used diagnostically to reveal understandings and misconceptions? Participants: The reviewed studies involved participants from college and K12 settings. Research Design: The reviewed studies all focused on learning processes and outcomes. Nearly all studies had high frequency process-tracing data such as concurrent think-alouds, moment-to-moment telemetry, or both. Participants interacted with an online task on an academic subject. The task typically spanned one or a few class periods, and the studies collectively examined relations among students' online behavior, cognitive processes (via think-alouds), and external measures of learning. Data Collection and Analysis: In the reviewed studies, students' online behavior was captured by instrumenting the system to capture and log interaction events. The more sophisticated approaches used a telemetry design based on the presumed cognitive processing occurring in the system. Findings: In general, measures derived from students' online behavior can be used (a) to decide when to intervene to influence learning processes (e.g., increased help seeking) and outcomes (e.g., improved course grades) and (b) as proxy measures of cognitive processing, understanding, and misconceptions. Conclusions: In the coming years, multiple levels of data will be fused to better understand the student, including static data such as demographics, low-frequency data such as interactions within a learning management system, and high frequency data such as moment-to-moment interactions in a digital app. As education enters the era of big data and transmedia-based learning, data of and for an individual will power new applications to realize the promise of personalized instruction.
Teachers College, Columbia University. P.O. Box 103, 525 West 120th Street, New York, NY 10027. Tel: 212-678-3774; Fax: 212-678-6619; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Elementary Secondary Education; Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A