NotesFAQContact Us
Search Tips
ERIC Number: ED537255
Record Type: Non-Journal
Publication Date: 2008-Mar
Pages: 30
Abstractor: As Provided
Reference Count: 17
Evaluating the Impacts of Item Exposure Procedures on Ability Estimates in CAT When Items are Disclosed
He, Wei; Reckase, Mark
Online Submission, Paper prepared for the National Council on Measurement in Education (NCME) Annual Meeting (New York City, NY, Mar 24-28, 2008)
Test security has been a concern for computerized adaptive tests (CAT) due to the nature of continuous testing. This concern becomes unprecedentedly severe with increasingly easy access to the World-Wide-Web where some examinees post on the internet their recollections of items they are administered, leaving future examinees with opportunities to get their scores inflated by having pre-knowledge of the items that appear in their tests. This study investigated, given some items in the item pool are disclosed, which item exposure control procedure results in minimum negative effect on ability estimate. The disclosed items in this study are defined as items released to the public or intentionally memorized by the test-takers and shared with future examinees and items to which examinees can respond correctly. The results from both simulation and statistical hypothesis testing are in consistent with Schnipke & Scrams (1999) in terms that the presence of disclosed items can inflate the final ability estimates in the context of CATs by an unacceptably high degree, especially for the low-ability examinees. Among all item exposure procedures under study, the a-stratified test with b-blocking, i.e., a design that stratifies the item pool but takes care of the relationship between item difficulty and discrimination parameters, performs the best. (Contains 18 tables, 3 figures, and 4 footnotes.)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A