NotesFAQContact Us
Search Tips
ERIC Number: ED542221
Record Type: Non-Journal
Publication Date: 2013
Pages: 27
Abstractor: As Provided
Reference Count: 19
A Comparison of Four Item-Selection Methods for Severely Constrained CATs
He, Wei; Diao, Qi; Hauser, Carl
Online Submission, Paper presented at the National Council on Measurement in Education (NCME) Annual Conference (San Francisco, CA, Apr 26-30, 2013)
This study compares the four existing procedures handling the item selection in severely constrained computerized adaptive tests (CAT). These procedures include weighted deviation model (WDM), weighted penalty model (WPM), maximum priority index (MPI), and shadow test approach (STA). Severely constrained CAT refer to those adaptive tests seeking to meet a complex set of content constraints simultaneously, acknowledging that an item usually carries multiple attributes that are inclusive to each other. In addition, two modified versions of the MPI procedure are introduced to deal with the situation in which the priority indices for all eligible items are zero. Given the item pool characteristic and the adaptive model within which this study is conducted, the results indicate that the shadow test approach, among all candidate methods, works the best in terms of measurement accuracy and constraint management, except that it makes the poorest use of items. All heuristic approaches do not differ significantly from each other in terms of measurement accuracy and constraint management at the lower bound level. However, the WPM method appears to perform considerably better in overall constraint management than both WDM and MPI methods. Regarding the two modified MPI procedures, the M2_MPI (i.e., the one assuming "move at its own pace" for each constraint) appears to perform better than the M1_MPI (i.e., the one assuming "move at the same pace" for all constraints) in overall constraint management. Regarding the three variations of WPM procedure, the WPM_fixed (1), i.e., the one adopting different weights to calculate content and item information penalty values, works better than other two variations. Limitations and further research directions are also discussed. (An appendix contains 4 tables and 7 figures.)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A