ERIC Number: EJ1217567
Record Type: Journal
Publication Date: 2019-Jun
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0022-0655
EISSN: N/A
Item Response Models for Multiple Attempts with Incomplete Data
Bergner, Yoav; Choi, Ikkyu; Castellano, Katherine E.
Journal of Educational Measurement, v56 n2 p415-436 Sum 2019
Allowance for multiple chances to answer constructed response questions is a prevalent feature in computer-based homework and exams. We consider the use of item response theory in the estimation of item characteristics and student ability when multiple attempts are allowed but no explicit penalty is deducted for extra tries. This is common practice in online formative assessments, where the number of attempts is often unlimited. In these environments, some students may not always answer-until-correct, but may rather terminate a response process after one or more incorrect tries. We contrast the cases of graded and sequential item response models, both unidimensional models which do not explicitly account for factors other than ability. These approaches differ not only in terms of log-odds assumptions but, importantly, in terms of handling incomplete data. We explore the consequences of model misspecification through a simulation study and with four online homework data sets. Our results suggest that model selection is insensitive for complete data, but quite sensitive to whether missing responses are regarded as informative (of inability) or not (e.g., missing at random). Under realistic conditions, a sequential model with similar parametric degrees of freedom to a graded model can account for more response patterns and outperforms the latter in terms of modelĀ fit.
Descriptors: Models, Item Response Theory, Homework, Computer Assisted Instruction, Computer Assisted Testing, Formative Evaluation, Simulation, Goodness of Fit, Item Analysis
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A