ERIC Number: ED475144
Record Type: Non-Journal
Publication Date: 2002-Apr
Reference Count: N/A
Tolerable Variation in Item Parameter Estimates.
Rizavi, Saba; Way, Walter D.; Davey, Tim; Herbert, Erin
The purpose of this study was to investigate and to quantify the tolerable error in item parameter estimates for different sets of items used in computer-based testing. The study examined items that were administered repeatedly to different examinee samples over time, examining items that were administered linearly in a fixed order each time they were used and items that appeared in different adaptive item testing pools each time they were used. In the first study, two sets of items from a high-stakes test, one set from the verbal test and one set from the quantitative test, were chosen, with sample sizes varying from 627 to 2,305 for the quantitative measure and 2,284 for the verbal measure. The second part of the study used items from another high stakes admissions test using 30 items with sample sizes larger than 500. Linearly administered items in a high-stakes testing program exhibited remarkably small variation in parameter estimates over repeated calibrations. Results also indicated that context effects played a more significant role in adaptive item parameters when comparisons were made to the parameters that were obtained from paper-and-pencil testing. This suggests that, whenever feasible, the parameter estimates obtained from paper-and-pencil administrations be replaced with computer-based testing calibrated parameters. Two appendixes contain the tables and figures that show sample sizes and parameter estimates. (Contains 4 tables, 18 figures, and 25 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.