NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1059779
Record Type: Journal
Publication Date: 2015
Pages: 25
Abstractor: As Provided
Reference Count: 26
ISSN: ISSN-1056-3997
The Effects of Rater Training on Inter-Rater Agreement
Pufpaff, Lisa A.; Clarke, Laura; Jones, Ruth E.
Mid-Western Educational Researcher, v27 n2 p117-141 2015
This paper addresses the effects of rater training on the rubric-based scoring of three preservice teacher candidate performance assessments. This project sought to evaluate the consistency of ratings assigned to student learning outcome measures being used for program accreditation and to explore the need for rater training in order to increase rater agreement. There were three phases during this project: (1) authentic student work was rated by department faculty members in the absence of rubric training; (2) faculty were then trained to administer rubric scoring guides; and (3) additional student work was rated by faculty after training. Inter-rater agreement was calculated pre- and post-rater training, using side-by-side comparisons. Little to no improvement in rater agreement was seen post-training. Implications and future research needs for rater training in the application of rubrics are discussed.
Mid-Western Educational Research Association. P.O. Box 34421, Chicago, IL 60634-0421. Tel: 419-372-7401; Fax: 419-372-2828; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A