ERIC Number: ED352388
Record Type: Non-Journal
Publication Date: 1992-Jul
Reference Count: N/A
Performance of the Mantel-Haenszel Statistic and the Standardized Difference in Proportions Correct When Population Ability Distributions Are Incongruent.
Spray, Judith A.; Miller, Timothy R.
A popular method of analyzing test items for differential item functioning (DIF) is to compute a statistic that conditions samples of examinees from different populations on an estimate of ability. This conditioning or matching by ability is intended to produce an appropriate statistic that is sensitive to true differences in item functioning, provided the ability estimate adequately reflects a comparable level of the true ability for these populations. If the observed or number-correct score is used as a conditioning or grouping variable, a problem exists whenever examinees from two different populations are matched on the same level of the observed test score, but actually have quite different levels of the unobserved ability. This occurs whenever the distributions of true abilities for the populations of interest are incongruent or non-overlapping. This situation was investigated in a series of computer simulations using 2,000 White and 2,000 African American examinees and item responses from previous administrations of the American College Testing Mathematics Usage Test. Results indicate that the magnitude of the problem, in terms of being able to detect true DIF with moderate sample sizes when ability distributions are incongruent, may not be that serious for tests that are, on the average, free of DIF. One table and four graphs support the discussion. (Author/SLD)
Descriptors: Blacks, College Entrance Examinations, Comparative Testing, Computer Simulation, Equations (Mathematics), Estimation (Mathematics), Item Bias, Mathematical Models, Mathematics Tests, Racial Bias, Racial Differences, Scores, Statistical Distributions, Test Items, Testing Problems, Whites
ACT Research Report Series, P.O. Box 168, Iowa City, IA 52243.
Publication Type: Reports - Evaluative; Reports - Research
Education Level: N/A
Authoring Institution: American Coll. Testing Program, Iowa City, IA.