NotesFAQContact Us
Search Tips
ERIC Number: ED498533
Record Type: Non-Journal
Publication Date: 2006-Apr
Pages: 52
Abstractor: Author
Reference Count: 16
Robustness of Ability Estimation to Multidimensionality in CAST with Implications to Test Assembly
Zhang, Yanwei; Nandakumar, Ratna
Online Submission, Paper Presented at the Annual Conference of the National Council of Measurement in Education (San Francisco, CA, April, 2006)
Computer Adaptive Sequential Testing (CAST) is a test delivery model that combines features of the traditional conventional paper-and-pencil testing and item-based computerized adaptive testing (CAT). The basic structure of CAST is a panel composed of multiple testlets adaptively administered to examinees at different stages. Current applications of CAST reply on the item response theory (IRT) and assume a unidimensional IRT model for scoring. This study evaluated the robustness of CAST when tests were constructed, administered, and scored by a unidimensional IRT model but item responses were multidimensional. Various conditions of multidimensionality were simulated in item pools, as well as different levels of content misclassification through manipulation of the correspondence between content area and dimension of items. An automated test assembly (ATA) process constructed CAST panels from the item pools, each representing a unique combination of multidimensionality and content misclassification. Administration of the panels was simulated and multidimensional response data were scored by the unidimensional IRT model. The ability scores, routing decisions, and pass-fail decisions were evaluated against "true" ability scores and decisions to assess the impacts of multidimensionality and content misclassification. Results showed that, when multidimensionality was mild as measured by the angle distance between item clusters, unidimensional ability estimates and routing decisions were not sensitive to the level of content misclassification in item pools. (Contains 8 tables and 14 figures.)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: California Achievement Tests