ERIC Number: EJ1168498
Record Type: Journal
Publication Date: 2017-Jun
Pages: 27
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
An Information-Correction Method for Testlet-Based Test Analysis: From the Perspectives of Item Response Theory and Generalizability Theory. Research Report. ETS RR-17-27
Li, Feifei
ETS Research Report Series, Jun 2017
An information-correction method for testlet-based tests is introduced. This method takes advantage of both generalizability theory (GT) and item response theory (IRT). The measurement error for the examinee proficiency parameter is often underestimated when a unidimensional conditional-independence IRT model is specified for a testlet dataset. By using a design effect ratio composed of random variances that can be easily derived from GT analysis, it becomes possible to adjust the underestimated measurement error from the unidimensional IRT models to a more appropriate level. In this paper, it is demonstrated how the information-correction method can be implemented in the context of a testlet design. Also, through the simulation study, it is shown that the underestimated measurement errors from IRT estimates can be adjusted to the appropriate level despite the varying magnitudes of local item dependence (LID), testlet length, balance of testlet length, and number of item parameters in the model. The real data example provides more details about when and how the information-correction method should be used in a test analysis. Estimation by the information-correction method should be adequate for practical work, given the robustness of the variance ratio.
Descriptors: Item Response Theory, Generalizability Theory, Tests, Error of Measurement, Test Items, Methods, Test Theory, Test Construction, Test Length, Reading Tests, Grade 3, Statistical Analysis, Factor Analysis
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 3
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A