NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED490661
Record Type: Non-Journal
Publication Date: 2005-Oct-14
Pages: 20
Abstractor: Author
Reference Count: 16
ISBN: N/A
ISSN: N/A
Free-Marginal Multirater Kappa (multirater K[free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa
Randolph, Justus J.
Online Submission, Paper presented at the Joensuu Learning and Instruction Symposium (Joensuu, Finland, Oct 14-15, 2005)
Fleiss' popular multirater kappa is known to be influenced by prevalence and bias, which can lead to the paradox of high agreement but low kappa. It also assumes that raters are restricted in how they can distribute cases across categories, which is not a typical feature of many agreement studies. In this article, a free-marginal, multirater alternative to Fleiss' multirater kappa is introduced. Free-marginal Multirater Kappa (multirater K[free]), like its birater free-marginal counterparts (PABAK, S, RE, and K[m],) is not influenced by kappa and is appropriate for the typical agreement study, in which raters' distributions of cases into categories are not restricted. Recommendations for the proper use of multirater K[free] are included. (Contains 5 equations, 3 tables, and 3 figures.)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A