NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED476471
Record Type: Non-Journal
Publication Date: 2003-May-12
Pages: 32
Abstractor: N/A
Reference Count: N/A
ISBN: N/A
ISSN: N/A
Calibration of Automatically Generated Items Using Bayesian Hierarchical Modeling.
Johnson, Matthew S.; Sinharay, Sandip
For complex educational assessments, there is an increasing use of "item families," which are groups of related items. However, calibration or scoring for such an assessment requires fitting models that take into account the dependence structure inherent among the items that belong to the same item family. C. Glas and W. van der Linden (2001) suggest a Bayesian hierarchical model to analyze data involving item families with multiple choice items. This paper extends the model to take into account item families with constructed response items, and designs a Markov chain Monte Carlo algorithm for the Bayesian estimation of model parameters. The hierarchical model, which accounts for the dependence structure inherent among the items, implicitly defines the "family response function" (FRF) for the score categories. This paper suggests a way to combine the FRFs over the score categories to obtain a "family score function" (FSF), which is a quick graphical summary of the expected score of an individual with a certain ability to an item randomly generated from an item family. The paper also suggests a method for the Bayesian estimation of the FRF and FSF. This work is a significant step towards building a tool to analyze data involving item families, and it may be very useful practically, for example, in automatic item generation systems that create tests involving item families. (Contains 1 table, 10 figures, and 25 references.) (Author/SLD)
Publication Type: Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A