NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ788438
Record Type: Journal
Publication Date: 2008
Pages: 9
Abstractor: Author
Reference Count: 17
ISSN: ISSN-0731-1745
Effects of Assigning Raters to Items
Sykes, Robert C.; Ito, Kyoko; Wang, Zhen
Educational Measurement: Issues and Practice, v27 n1 p47-55 Spr 2008
Student responses to a large number of constructed response items in three Math and three Reading tests were scored on two occasions using three ways of assigning raters: single reader scoring, a different reader for each response (item-specific), and three readers each scoring a rater item block (RIB) containing approximately one-third of a student's responses. Multiple group confirmatory factor analyses indicated that the three types of total scores were most frequently tau-equivalent. Factor models fitted on the item responses attributed differences in scores to correlated ratings incurred by the same reader scoring multiple responses. These halo effects contributed to significantly increased single reader mean total scores for three of the tests. The similarity of scores for item-specific and RIB scoring suggests that the effect of rater bias on an examinee's set of responses may be minimized with the use of multiple readers though fewer than the number of items.
Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A