ERIC Number: ED343931
Record Type: RIE
Publication Date: 1990-Mar
Cognitive Assessment of Subject Matter: Understanding the Marriage of Psychological Theory and Educational Policy in Achievement Testing.
Baker, Eva L.; And Others
As part of a project on assessing deep understanding of subject matter, a study was conducted to assess students' knowledge of history by focusing on essay writing. Reading a provided text was incorporated into the assessment procedure. Ratings from five high school history teachers were compared with those of four English teachers for 85 essays from 11th-grade advanced placement students in a suburban high school. No significant differences in student performance were found for the two text passages or question type (brief or extended) in ratings of either group. English and history teachers were looking at student papers in fundamentally similar ways, but scoring criteria were not adequate in that they focused on what teachers said they valued, rather than what they actually would write themselves. Using an expert-novice model, new questions were developed, and the new test was administered to 250 11th graders and scored under new scoring rubrics. Higher interrater reliabilities encouraged researchers to train four history teachers in the new scoring approach. Ongoing research is examining the utility of the new scoring focus for high school students in two school districts. The study illustrates the complexity of developing new and useful assessments. Eight tables summarize test development principles, and a 30-item list of references is included. (SLD)
Descriptors: Achievement Tests, Advanced Placement Programs, Cognitive Measurement, Comparative Testing, Educational Policy, English, Essay Tests, Evaluators, Grade 11, High School Students, High Schools, History, Interrater Reliability, Knowledge Level, Reading Comprehension, Scoring, Secondary School Teachers, Standardized Tests, Test Construction, Writing Tests
CSE Dissemination Office, UCLA Graduate School of Education, 405 Hilgard Avenue, Los Angeles, CA 90024-1521.
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.