NotesFAQContact Us
Search Tips
ERIC Number: ED453288
Record Type: Non-Journal
Publication Date: 2001-Apr-13
Pages: 33
Abstractor: N/A
Reference Count: N/A
Do Raters Demonstrate Halo Error When Scoring a Series of Responses?
Ridge, Kirk
This study investigated whether raters in two different training groups would demonstrate halo error when each rater scored all five responses to five different mathematics performance-based items from each student. One group of 20 raters was trained by an experienced scoring director with item-specific scoring rubrics and the opportunity to practice scoring and receive feedback. A second group of 20 raters trained themselves using a generic scoring rubric without the opportunity to practice. After training, all raters scored the same 500 student responses, into which were embedded manufactured student response scenarios that were designed to induce halo error. The results indicate that the director-trained raters were more accurate than the self-trained raters on low-scoring responses, but there were no significant differences between the groups in rater halo error. A followup study was conducted using a third group of raters who were not trained and were instructed to rate the papers according to perceived quality. The results of the analysis indicated a significant difference between this group and the director-trained group in rater halo error. (Contains 2 figures, 9 tables, and 15 references.) (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A