ERIC Number: ED454293
Record Type: RIE
Publication Date: 2000-Nov
On the "Exchangeability" of Hands-On and Computer-Simulated Science Performance Assessments. CSE Technical Report.
Rosenquist, Anders; Shavelson, Richard J.; Ruiz-Primo, Maria Araceli
Inconsistencies in scores from computer-simulated and "hands-on" science performance assessments have led to questions about the exchangeability of these two methods in spite of the highly touted potential of computer-simulated performance assessment. This investigation considered possible explanations for students' inconsistent performances: (1) inadequate exposure to computers and simulations; (2) differential views of computer simulated (two-dimensional icons) and hands-on tasks; (3) different methods tapping different aspects of achievement; (4) partial or incomplete knowledge; and (5) a combination of partial knowledge and method differences. A series of short studies with fifth graders (usually approximately 17 students) used the Electric Mysteries computer-based assessment. The first explanation was ruled out because of students' familiarity with computers and simulations in the classroom. The second explanation was ruled out using talk-aloud data, randomized experiments, and student questionnaire responses. The third explanation was also considered not tenable because correlations did not indicate that different methods tapped different areas of achievement. To explore the remaining explanations, the performance of 40 high school physics students (experts) was compared with that of elementary school students in an earlier study. The "experts" were far from expert, and were, in fact, no more "expert" than the elementary school students. The possible explanation for the lack of exchangeability between computer-simulated and hands-on performance assessments has been narrowed to one of two choices: partial knowledge or the interaction of partial knowledge with method. (Contains 6 figures, 1 table, and 11 references.) (SLD)
Descriptors: Computer Assisted Testing, Computer Simulation, Elementary School Students, Hands on Science, High School Students, High Schools, Intermediate Grades, Knowledge Level, Performance Based Assessment, Science Education, Sciences, Test Format, Test Results
National Center for Research on Evaluation, CRESST/CSE, Graduate School of Education & Information Studies, University of California, Los Angeles, CA 90095-1522. Tel: 310-206-1532.
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.; California Univ., Los Angeles. Center for the Study of Evaluation.