ERIC Number: ED330719
Record Type: RIE
Publication Date: 1991-Apr
Reference Count: N/A
Confidence in Pass/Fail Decisions for Computer Adaptive and Paper and Pencil Examinations.
Bergstrom, Betty A.; Lunz, Mary E.
The level of confidence in pass/fail decisions obtained with computer adaptive tests (CATs) was compared to decisions based on paper-and-pencil tests. Subjects included 645 medical technology students from 238 educational programs across the country. The tests used in this study constituted part of the subjects' review for the certification examination. Examinees took a variable length CAT and two fixed-length paper-and-pencil tests. The CAT, which was based on a pool of 726 Rasch calibrated items, was designed to determine whether an examinee's estimated ability level is above or below a pre-established pass/fail point. Short and long paper-and-pencil tests were used, containing 109 items and 189 items, respectively, selected from the CAT pool of 726 items. Only 600 examinees participated in the long paper-and-pencil test. The CAT was stopped when the examinee measure was 1.3 times the error of measure above or below the pass/fail point (one tailed 90% confidence interval) or when a maximum test length was reached. Results show that greater confidence in the accuracy of pass/fail decisions is obtained for more examinees when the CAT implements a 90% confidence stopping rule than with paper-and-pencil tests of comparable length. Using a confidence interval stopping rule with a CAT allowed each examinee to take a minimum number of items and insures that pass/fail decisions are made with a high level of confidence. Two tables and four figures are included. (TJH)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Confidence Intervals (Statistics); Paper and Pencil Tests; Rasch Model
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Chicago, IL, April 3-7, 1991).