ERIC Number: EJ1026121
Record Type: Journal
Publication Date: 2014-Jun
Abstractor: As Provided
Reference Count: 21
Item Pool Design for an Operational Variable-Length Computerized Adaptive Test
He, Wei; Reckase, Mark D.
Educational and Psychological Measurement, v74 n3 p473-494 Jun 2014
For computerized adaptive tests (CATs) to work well, they must have an item pool with sufficient numbers of good quality items. Many researchers have pointed out that, in developing item pools for CATs, not only is the item pool size important but also the distribution of item parameters and practical considerations such as content distribution and item exposure issues. Yet, there is little research on how to design item pools to have those desirable features. The research reported in this article provided step-by-step hands-on guidance on the item pool design process by applying the bin-and-union method to design item pools for a large-scale licensure CAT employing complex adaptive testing algorithm with variable test length, a decision based on stopping rule, content balancing, and exposure control. The design process involved extensive simulations to identify several alternative item pool designs and evaluate their performance against a series of criteria. The design output included the desired item pool size and item parameter distribution. The results indicate that the mechanism used to identify the desirable item pool features functions well and that two recommended item pool designs would support satisfactory performance of the operational testing program.
Descriptors: Item Banks, Test Length, Computer Assisted Testing, Adaptive Testing, Test Construction, Licensing Examinations (Professions), Item Response Theory, Bayesian Statistics, Computation, Maximum Likelihood Statistics, Ability, Difficulty Level, Test Items, Accuracy, Test Bias
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: firstname.lastname@example.org; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A