ERIC Number: ED368777
Record Type: RIE
Publication Date: 1994-Apr
Reference Count: N/A
Learning To Rate Essays: A Study of Scorer Cognition.
Wolfe, Edward W.; Feltovich, Brian
This paper presents a model of scored cognition that incorporates two types of mental models: models of performance (i.e., the criteria for judging performance) and models of scoring (i.e., the procedural scripts for scoring an essay). In Study 1, six novice and five experienced scorers wrote definitions of three levels of a 6-point holistic scoring rubric at two times during a scoring project for a large-scale standardized writing assessment. Given practice and experience (3 days) in scoring, models of performance used by novices began to approximate those used by experienced scores. In Study 2, three better experienced scorers and three poorer experienced scorers engaged in a think-aloud task on three essays. This study revealed that better scorers stop more often while reading essays to comment and that the reading styles of better scorers are more consistent as a group. Better scorers are also more likely to make nonevaluative comments about the test itself or the writer, and they are more consistent in their use of content categories when discussing essay characteristics and more likely to focus on complex and abstract qualities (e.g., writer's voice or content development) than were poorer scorers. Six tables, three fugures are included. (Contains 25 references.) (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Experts; Mental Models
Note: Paper presented at the Annual Meeting of the American Educational Research Association (New Orleans, LA, April 4-8, 1994).