ERIC Number: ED462422
Record Type: Non-Journal
Publication Date: 1999-Jul-30
Reference Count: N/A
Practical Implications of Test Dimensionality for Item Response Theory Calibration of the Medical College Admission Test. MCAT Monograph.
Childs, Ruth A.; Oppler, Scott H.
The use of item response theory (IRT) in the Medical College Admission Test (MCAT) testing program has been limited. This study provides a basis for future IRT analyses of the MCAT by exploring the dimensionality of each of the MCAT's three multiple-choice test sections (Verbal Reasoning, Physical Sciences, and Biological Sciences) and the implications of that dimensionality for IRT calibration and scoring. Three sets of data were used: an MCAT form administered on 2 occasions (16,520 and 3,638 observations) and a second, content-equivalent form administered on the second occasion (12,625 observations). Dimensionality analyses suggested that while the items within each of the science test sections are not completely homogeneous, nor are they measuring distinct constructs corresponding to the disciplines. As is consistent with this finding, when confirmatory factor analyses were performed, the Biological Sciences disciplines were found to be correlated about 0.85, and the Physical Sciences about 0.95. Comparison of the results of the IRT calibration and scoring of the data by test and by discipline within the test suggests that the tests do not deviate sufficiently from unidimensionality to require that the IRT item calibrations be performed separately by discipline within the test. The use of these results to inform future IRT analyses of the MCAT depends in large part on policy decisions about future directions of the MCAT program. (Contains 17 tables, 3 figures, and 20 references.) (SLD)
Descriptors: College Entrance Examinations, Higher Education, Item Response Theory, Medical Education, Scoring
Association of American Medical Colleges, Section for the Medical College Admission Test, 2450 N Street, NW, Washington, DC 20037. Tel: 202-828-0400; Fax: 202-828-1125; Web site: http://www.aamc.org/mcat.
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Association of American Medical Colleges, Washington, DC.
Authoring Institution: American Institutes for Research, Washington, DC.
Identifiers - Assessments and Surveys: Medical College Admission Test