NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ793269
Record Type: Journal
Publication Date: 2004
Pages: 13
Abstractor: Author
Reference Count: 51
ISSN: ISSN-1534-5084
A Comparison of Two Methods of Determining Interrater Reliability
Fleming, Judith A.; Taylor, Janeen McCracken; Carran, Deborah
Assessment for Effective Intervention, v29 n2 p39-51 2004
This article offers an alternative methodology for practitioners and researchers to use in establishing interrater reliability for testing purposes. The majority of studies on interrater reliability use a traditional methodology where by two raters are compared using a Pearson product-moment correlation. This traditional method of estimating interrater reliability uses a correlation of the two raters' scores for the same subject. This study used an observer-rater paradigm in which 20 observers' scores were compared to the scores of an expert rater. Results bring into question the common practice of examining interrater reliability. The correlation of two raters may not be a reliable way of determining if every evaluator using the instrument will obtain scores that accurately report the developmental status of young children. (Contains 6 tables.)
PRO-ED, Inc. Available from: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A