NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1011155
Record Type: Journal
Publication Date: 2013-Jun
Pages: 20
Abstractor: As Provided
Reference Count: 35
ISSN: ISSN-0013-1644
Interrater Agreement Evaluation: A Latent Variable Modeling Approach
Raykov, Tenko; Dimitrov, Dimiter M.; von Eye, Alexander; Marcoulides, George A.
Educational and Psychological Measurement, v73 n3 p512-531 Jun 2013
A latent variable modeling method for evaluation of interrater agreement is outlined. The procedure is useful for point and interval estimation of the degree of agreement among a given set of judges evaluating a group of targets. In addition, the approach allows one to test for identity in underlying thresholds across raters as well as to identify possibly aberrantly evaluating judges. A measure of interrater agreement is proposed, which is related to popular indexes of interrater reliability for observed variables and composite reliability. The outlined method also permits the examination of underlying common sources of ratings variability, provides a useful complement to the literature on interrater agreement with manifest measures, and relaxes some of its assumptions. The procedure is illustrated with numerical data. (Contains 1 table and 3 notes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A