NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED523778
Record Type: Non-Journal
Publication Date: 2010-Sep
Pages: 42
Abstractor: As Provided
Reference Count: 24
ISBN: N/A
ISSN: N/A
Application of a General Polytomous Testlet Model to the Reading Section of a Large-Scale English Language Assessment. Research Report. ETS RR-10-21
Li, Yanmei; Li, Shuhong; Wang, Lin
Educational Testing Service
Many standardized educational tests include groups of items based on a common stimulus, known as "testlets". Standard unidimensional item response theory (IRT) models are commonly used to model examinees' responses to testlet items. However, it is known that local dependence among testlet items can lead to biased item parameter estimates when using standard IRT models, and to overestimated reliability. In this study, a general polytomous testlet model was proposed to account for local dependence in testlet-based tests that contain both dichotomously and polytomously scored items. The proposed model and a standard IRT model were fit to simulated data and several real data sets from the reading sections of a large-scale English-language test, and model fit was evaluated. Item parameters and test information obtained from the two models were compared to check the impact of local item dependence. In addition, a multidimensional IRT model with simple structure was also fit to the real data sets. Results based on both simulated and real data suggested that local dependence had a small impact on item parameter estimates and a relatively larger impact on test information and reliability. It was also found that the multidimensional IRT model with simple structure fit the real data sets better than the general polytomous testlet model and the standard IRT model did. An Example of SAS Code Used for Estimating the General Polytomous Testlet Model, the 2PL/GPCM Model, and the Multidimensional IRT Model with Simple Structure is appended. (Contains 3 figures and 10 tables.)
Educational Testing Service. Rosedale Road Mailstop 19R, Princeton, NJ 08541-0001. Tel: 609-921-9000; Fax: 609-734-5410; Web site: http://www.ets.org
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Educational Testing Service