NotesFAQContact Us
Search Tips
ERIC Number: ED532068
Record Type: Non-Journal
Publication Date: 2012-Feb
Pages: 33
Abstractor: As Provided
Reference Count: 81
Measuring and Promoting Inter-Rater Agreement of Teacher and Principal Performance Ratings
Graham, Matthew; Milanowski, Anthony; Miller, Jackson
Online Submission
As states, districts, and schools transition toward more rigorous educator evaluation systems, they are placing additional weight on judgments about educator practice. Since teacher and principal observation ratings inherently rely on evaluators' professional judgment, there is always a question of how much the ratings depend on the particular evaluator rather than the educator's actual performance. To help states, districts, and schools measure and maximize consistency of evaluator observation ratings, this paper: (1) draws a distinction between inter-rater reliability and interrater agreement, (2) reviews methods for calculating inter-rater reliability and agreement, (3) reviews thresholds for inter-rater agreement scores, and (4) identifies practices that can improve inter-rater reliability and inter-rater agreement. Two appendixes present: (1) More on Intra-Class Correlations; and (2) Frame-of-Reference Training Outline. (Contains 7 footnotes and 7 tables.) [This report was produced by the Center for Educator Compensation Reform (CECR).]
Publication Type: Reports - Descriptive
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: Department of Education (ED)
Authoring Institution: N/A