NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ779498
Record Type: Journal
Publication Date: 2002
Pages: 12
Abstractor: Author
Reference Count: 0
ISSN: ISSN-0146-6216
The Information in Multiple Ratings
Bock, R. Darrell; Brennan, Robert L.; Muraki, Eiji
Applied Psychological Measurement, v26 n4 p364-375 2002
In assessment programs where scores are reported for individual examinees, it is desirable to have responses to performance exercises graded by more than one rater. If more than one item on each test form is so graded, it is also desirable that different raters grade the responses of any one examinee. This gives rise to sampling designs in which raters are nested within items. These designs lead to simple methods for estimating variance components owing to examinees and to interactions of examinees by items and examinees by raters within items. The authors review here some useful results from generalizability analysis based on these estimates and show that they may be used to correct the item response information functions and standard errors for conditional dependence of multiple ratings. Examples based on data from two performance testing studies are presented.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A