ERIC Number: ED159226
Record Type: RIE
Publication Date: 1978-Mar-27
Reference Count: 0
A Monte Carlo Comparison of Phi and Kappa as Measures of Criterion-Referenced Reliability.
Reid, Jerry B.; Roberts, Dennis M.
Comparisons of corresponding values of phi and kappa coefficients were made for 270 instances of data generated by a Monte Carlo technique to simulate a test-retest situation. Data were generated for distributions with the same mean but three different levels of standard deviation, standard error of measurement and cutting score. Ten samples of 300 cases were used for each of the 27 crossed conditions. Comparisons of the corresponding phi and kappa values showed that none of the pairs of coefficients had differences larger than .04 and 98 percent had differences less than .01. Kappa is frequently recommended as the appropriate indicator for the reliability of criterion referenced tests, but since phi is a widely known correlation coefficient, its use may be preferable. (Author/CTM)
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Kappa Coefficient; Phi Coefficient
Note: Paper presented at the Annual Meeting of the American Educational Research Association (62nd, Toronto, Ontario, Canada, March 27-31, 1978)