NotesFAQContact Us
Search Tips
ERIC Number: ED411297
Record Type: Non-Journal
Publication Date: 1997-Mar
Pages: 32
Abstractor: N/A
Reference Count: N/A
Maintaining Item and Test Security in a CAT Environment: A Simulation Study. Laboratory of Psychometric and Evaluative Research Report No. 309.
Patsula, Liane N.; Steffen, Mandred
One challenge associated with computerized adaptive testing (CAT) is the maintenance of test and item security while allowing for daily testing. An alternative to continually creating new pools containing an independent set of items would be to consider each CAT pool as a sample of items from a larger collection (referred to as a VAT) rather than as a permanent collection of items. This research investigated the viability of creating overlapping CAT pools from a static VAT by sampling items with replacement. The VAT used in this simulation consisted of approximately 5,000 precalibrated verbal multiple-choice items. Item pools were created, and simulations were performed to configure each of these item collections into CAT pools and to estimate the proportion of examinees at each of 13 ability levels that were expected to see each item. Probabilities of administration were computed for each item and different numbers of examinees and item pools were simulated. As the number of pools increased, their quality decreased, and the proportion of items reaching the retirement threshold for test items decreased. As more pools were created, the potential for control increased. Results support the use of a VAT system as a way of drawing a compromise between test quality and test security. (Contains five figures, six tables, and five references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A