ERIC Number: ED190603
Record Type: RIE
Publication Date: 1980-Apr
Reference Count: 0
A Comparison of Decision-Making Methods for Criterion-Referenced Tests.
Haladyna, Tom; Roid, Gale
The problems associated with misclassifying students when pass-fail decisions are based on test scores are discussed. One protection against misclassification is to set a confidence interval around the cutting score. Those whose scores fall above the interval are passed; those whose scores fall below the interval are failed; and those whose scores fall within the interval are retained for brief remedial instruction. Six methods which may be used to set the confidence level include: (1) Novick and Jackson's Bayesian method, which uses a computer program for statistical analysis, Computer-Assisted Data Analysis; (2) Millman's binomial method; (3) Shaycroft's variation of the binomial method; (4) Lord and Novick's classical approach; (5) Cronbach et. al.'s generalizability theory; and (6) Rasch's latent trait model. These six methods were empirically compared using criterion-referenced test score data from 80-item and 100-item tests of dental anatomy, as well as 68-item elementary school achievement tests. The interaction of the methods with passing standards and with test length was also investigated. While the results indicated promise for Bayesian and item-response (Rasch) theories, no single procedure proved to be more consistent in minimizing classification errors. (Author/GDC)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Binomial Coefficients; Classical Test Theory; Generalizability Theory
Note: Paper presented at the Annual Meeting of the American Educational Research Association (64th, Boston, MA, April 7-11, 1980).