PDF pending restoration
ERIC Number: ED187745
Record Type: RIE
Publication Date: 1979-Feb
Reference Count: 0
Accuracy of Two Procedures for Estimating Reliability of Mastery Tests. Research Memorandum 79-1.
Hunyh, Hunyh; Saunders, Joseph C.
Comparisons were made among various methods of estimating the reliability of pass-fail decisions based on mastery tests. The reliability indices that are considered are p, the proportion of agreements between two estimates, and kappa, the proportion of agreements corrected for chance. Estimates of these two indices were made on the basis of repeated or alternate test administrations and also on the basis of a single administration with the assumption of a beta-binomial or negative-hypergeometric distribution. Across a variety of test score distributions, test lengths, and mastery (cutoff) scores, the beta-binomial estimates tended to underestimate the corresponding population values. The percent of bias was small (about 2.5%) for p and somewhat larger (about 10%) for kappa. Both beta-binomial estimates had standard errors about one-half the size of the standard errors of estimates based on repeated test administrations. Though the beta-binomial estimates presumed equality of item difficulty, the data presented indicated that even gross departures from equality of item difficulty did not affect the amount of bias of the estimates. (Author/CTM)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC.
Authoring Institution: South Carolina Univ., Columbia.
Identifiers: Kappa Coefficient (Cohen)
Note: Paper presented at the Annual Meeting of the Eastern Educational Research Association (Kiawah Island, SC, February 22-24, 1979).