NotesFAQContact Us
Search Tips
ERIC Number: ED422393
Record Type: Non-Journal
Publication Date: 1997-Mar-26
Pages: 35
Abstractor: N/A
Reference Count: N/A
A Generalizability Approach To Evaluating the Reliability of Testlet-Based Test Scores.
Lee, Guemin; Frisbie, David A.
Previous studies have indicated that the reliability of test scores composed of testlets might be overestimated by conventional item-based reliability estimation methods (R. Thorndike, 1953; A. Anastasi, 1988; S. Sireci, D. Thissen, and H. Wainer, 1991; H. Wainer and D. Thissen, 1996). This study used generalizability theory to investigate the relative adequacy of reliability coefficients from test scores composed of testlets with a p x (I:H) random effects design, where persons are crossed with items nested within passages. The magnitude of overestimation of using Cronbach's coefficient alpha based on item scores in this situation was estimated to be about 0.04. The passage facet turns out to be more influential on reliability estimates than the item-within-passage facet. Given a fixed total number of items and a fixed number of passages, the variability of generalizability coefficients with varying number of items per passage is small (under 0.01). Therefore, manipulating the number of passages is a more productive way to obtain efficient measurement procedures than is manipulating the number of items within each passage. (Contains 7 tables, 3 figures, and 18 references.) (Author)
Publication Type: Numerical/Quantitative Data; Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A