ERIC Number: ED185076
Record Type: RIE
Publication Date: 1977-Dec
Reference Count: 0
Agreement Coefficients as Indices of Dependability for Domain-Referenced Tests. ACT Technical Bulletin No. 28.
Kane, Michael T.; Brennan, Robert L.
A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of these two indices are determined by assumptions about the nature of the agreement function or, equivalently, the nature of the loss function for the testing procedure. For example, indices discussed by Huynh, Subkoviak, and Swaminathan, Hambleton, and Algina employ a threshold agreement, or loss, function whereas, indices discussed by Brennan and Kane and Livingston employ a squared error loss function. Since all of these indices are discussed within a single general framework, the differences among them in their assumptions, properties, and uses can be exhibited clearly. For purposes of comparison, norm-referenced generalizability coefficients are also developed and discussed within this general framework. (Author/CTM)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Navy Personnel Research and Development Center, San Diego, CA.
Authoring Institution: American Coll. Testing Program, Iowa City, IA. Research and Development Div.
Identifiers: Domain Referenced Tests; Kappa Coefficient; Loss Function; Phi Coefficient