ERIC Number: ED462426
Record Type: Non-Journal
Publication Date: 2001-Dec
Reference Count: N/A
Effects of Local Item Dependence on the Validity of IRT Item, Test, and Ability Statistics. MCAT Monograph.
Zenisky, April L.; Hambleton, Ronald K.; Sireci, Stephen G.
Measurement specialists routinely assume examinee responses to test items are independent of one another. However, previous research has shown that many contemporary tests contain item dependencies and not accounting for these dependencies leads to misleading estimates of item, test, and ability parameters. In this study, methods for detecting local item dependence (LID) are reviewed, and the use of testlets to account for LID in context-dependent item sets is discussed. LID detection methods and testlet-based item calibrations are applied to data from a large-scale, high stakes admissions test, and the results are evaluated with respect to test score reliability and examinee proficiency estimation. Data were from two forms of the Medical College Admission Test (MCAT) for 8,494 and 8,026 examinees. Results suggest the presence of LID impacts estimation of examinee proficiency. The practical effects of the presence of LID on passage-based tests are discussed, as are issues regarding the calibration of context-dependent item sets using item response theory. (Contains 3 figures, 11 tables, and 30 references.) (Author/SLD)
Descriptors: Ability, College Applicants, College Entrance Examinations, Higher Education, Identification, Item Response Theory, Test Items, Validity
Association of American Medical Colleges, Section for the Medical College Admission Test, 2450 N Street, NW, Washington, DC 20037. Tel: 202-828-0400; Fax: 202-828-1125; Web site: http://www.aamc.org/mcat.
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Association of American Medical Colleges, Washington, DC.
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Medical College Admission Test