ERIC Number: EJ1073493
Record Type: Journal
Publication Date: 2015-Oct
Abstractor: As Provided
Reference Count: 60
Taking the Missing Propensity into Account When Estimating Competence Scores: Evaluation of Item Response Theory Models for Nonignorable Omissions
Köhler, Carmen; Pohl, Steffi; Carstensen, Claus H.
Educational and Psychological Measurement, v75 n5 p850-874 Oct 2015
When competence tests are administered, subjects frequently omit items. These missing responses pose a threat to correctly estimating the proficiency level. Newer model-based approaches aim to take nonignorable missing data processes into account by incorporating a latent missing propensity into the measurement model. Two assumptions are typically made when using these models: (1) The missing propensity is unidimensional and (2) the missing propensity and the ability are bivariate normally distributed. These assumptions may, however, be violated in real data sets and could, thus, pose a threat to the validity of this approach. The present study focuses on modeling competencies in various domains, using data from a school sample (N = 15,396) and an adult sample (N = 7,256) from the National Educational Panel Study. Our interest was to investigate whether violations of unidimensionality and the normal distribution assumption severely affect the performance of the model-based approach in terms of differences in ability estimates. We propose a model with a competence dimension, a unidimensional missing propensity and a distributional assumption more flexible than a multivariate normal. Using this model for ability estimation results in different ability estimates compared with a model ignoring missing responses. Implications for ability estimation in large-scale assessments are discussed.
Descriptors: Competence, Tests, Evaluation Methods, Adults, Responses, Item Response Theory, Performance, Ability, Measurement Techniques, Data Analysis, Error of Measurement, Statistical Inference, Computation, Probability, Scaling, Longitudinal Studies, Grade 9, Foreign Countries
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 9; Junior High Schools; Middle Schools; Secondary Education; High Schools
Authoring Institution: N/A
Identifiers - Location: Germany