ERIC Number: ED455285
Record Type: RIE
Publication Date: 2001-Mar
Making Sense of Data from Complex Assessments. CSE Technical Report.
Mislevy, Robert J.; Steinberg, Linda S.; Almond, Russell G.; Breyer, F. Jay; Johnson, Lynn
Advances in cognitive psychology deepen the understanding of how students gain and use knowledge and broaden the range of performances and situations researchers want to see to acquire evidence about students' developing knowledge. Advances in technology make it possible to capture more complex performances in assessment settings by including, as examples, simulation, interactivity, and extended responses. The challenge is making sense of the complex data that result. This paper concerns an evidence-centered approach to the design and analysis of complex assessments. It presents a design framework that incorporates integrated structures for modeling knowledge and skills, designing tasks, and extracting and synthesizing evidence. The ideas are illustrated in the context of a project with the Dental Interactive Simulation Corporation (DISC) in which problem solving in dental hygiene is assessed with computer-based simulations. After reviewing the substantive grounding of this effort, the paper describes the design rationale, statistical and scoring models, and operational structures for the DISC assessment prototype. Two appendixes contain an example of the use of Bayes Nets in assessment and a list of DISC evidence models. (Contains 11 figures and 26 references.) (Author/SLD)
Descriptors: Cognitive Psychology, Computer Simulation, Data Analysis, Dental Hygienists, Educational Assessment, Educational Improvement, Educational Technology, Problem Solving, Research Design, Technological Advancement
National Center for Research on Evaluation, CRESST/CSE, Graduate School of Education & Information Studies, University of California, Los Angeles, CA 90095-1522. Tel: 310-206-1532.
Publication Type: Reports - Descriptive
Education Level: N/A
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.; California Univ., Los Angeles. Center for the Study of Evaluation.
Note: For a related paper on assessment, see TM 033 137.