NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ893543
Record Type: Journal
Publication Date: 2010
Pages: 32
Abstractor: As Provided
Reference Count: 89
ISBN: N/A
ISSN: ISSN-1530-5058
A Monte Carlo Simulation Investigating the Validity and Reliability of Ability Estimation in Item Response Theory with Speeded Computer Adaptive Tests
Schmitt, T. A.; Sass, D. A.; Sullivan, J. R.; Walker, C. M.
International Journal of Testing, v10 n3 p230-261 2010
Imposed time limits on computer adaptive tests (CATs) can result in examinees having difficulty completing all items, thus compromising the validity and reliability of ability estimates. In this study, the effects of speededness were explored in a simulated CAT environment by varying examinee response patterns to end-of-test items. Expectedly, ability estimates became increasingly negatively biased as the CAT became more speeded, with the bias magnitude depending on the speededness condition. Ability estimates for higher-ability examinees were also influenced more than estimates for low- and middle-ability examinees with the realistic item pool. This finding is likely linked to test operational characteristics since these results were not replicated with an ideal item pool. In any case, the CAT is relatively robust to the speededness assumption, assuming that only a few items are speeded. Results also suggested that if test developers score the test at the point of speededness, ability estimates remain unbiased. This study's findings help inform test developers and test takers about the effects of test speededness within a CAT environment. Additional implications for educators and researchers are discussed. (Contains 2 tables, 3 figures and 2 notes.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A