ERIC Number: ED201667
Record Type: RIE
Publication Date: 1977
Reference Count: 0
Robinson's Measure of Agreement as a Parallel Forms Reliability Coefficient.
Willson, Victor L.
A major deficiency in classical test theory is the reliance on Pearson product-moment (PPM) correlation concepts in the definition of reliability. PPM measures are totally insensitive to first moment differences in tests which leads to the dubious assumption of essential tan-equivalence. Robinson proposed a measure of agreement that is sensitive to different test difficulty and gives a practical statistic to estimate reliability in the presence of known form variation in difficulty. Robinson's measure of agreement appears to be a useful alternative to the generalizability coefficient, as it provides a more conservative estimate of reliability under conditions of parallel form differences in mean. This is likely to be especially useful when examining inter rater reliability when internal consistency of the raters is poor. Robinson's measure does not seem advantageous for highly reliable parallel tests such as are encountered in standardized testing programs. A simulation study is presented to illustrate the degree of the coefficient's sensitivity to form difficulty variance. Robinson's measure of agreement and the intraclass correlation are computed for each simulation and their values are compared. (author/RL)
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Parallel Forms Reliability; Robinsons Measure of Agreement