ERIC Number: ED230617
Record Type: RIE
Publication Date: 1983-Apr
Reference Count: 0
Comparing Four Estimates of the Criterion-Referenced Standard for a Written Test.
Hughes, Francis P.
Four procedures were used to estimate a criterion-referenced standard for a multiple-choice examination developed by the National Board of Medical Examiners (NBME). Two experimental procedures, the NBME method and a modification of the Guerin method, and the Angoff and Ebel procedures were evaluated on the consistency of the estimates they yielded, the plausibility of the failure rates, and the standard-setters' confidence in their judgments. The NBME and modified Guerin procedures yielded the most consistent and least consistent estimates, respectively. The failure rates associated with the standards obtained using these procedures were higher than the failure rate associated with the test's norm-referenced standard, but only the failure rate associated with the modified Guerin procedure was obviously unacceptable. The standard-setters said it was difficult to judge the success rate of "minimally knowledgeable examinees" with the test questions, but even more difficult to make those judgments for the hypothetical classifications of items used with the Ebel procedure. The estimates obtained using three of the procedures were relatively consistent and the failure rates associated with them, although higher than the rate experienced with a norm referenced standard, were plausible. (Author/PN)
Publication Type: Speeches/Meeting Papers; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A
Identifiers: Angoff Methods; Ebel Method; Guerin Method; National Board of Medical Examiners; Standards for Educational and Psychological Tests
Note: Paper presented at the Annual Meeting of the American Educational Research Association (67th, Montreal, Quebec, April 11-15, 1983); Some figures may be marginally legible due to small print.