ERIC Number: ED412236
Record Type: Non-Journal
Publication Date: 1997-Mar
Reference Count: N/A
A Method To Compare Rater Severity across Several Administrations.
O'Neill, Thomas R.; Lunz, Mary E.
This paper illustrates a method to study rater severity across exam administrations. A multi-facet Rasch model defined the ratings as being dominated by four facets: examinee ability, rater severity, project difficulty, and task difficulty. Ten years of data from administrations of a histotechnology performance assessment were pooled and analyzed to establish a scale. Next, the 17 individual administrations were anchored to that scale and reanalyzed. The severity of the nine raters who graded most often were listed and plotted by administration. These plots show the consistency of each rater's level of severity. Results show that: (1) raters have an individual level of severity; (2) raters can usually maintain that level; and (3) some raters maintain their level of severity more consistently than others. The implications for equating performance assessments prospectively are discussed. (Contains 2 tables, 10 figures, and 5 references.) (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A