ERIC Number: ED346120
Record Type: RIE
Publication Date: 1992-Apr
Reference Count: 0
The Effects of Sample Size and Matching Strategy on Mantel-Haenszel and Logit DIF Procedures.
Schultz, Matthew T.; Geisinger, Kurt F.
Research efforts have established that the Mantel-Haenszel procedure (MHP) is an effective method for detecting the presence of test items exhibiting differential item functioning (DIF). While the MHP has been advocated for situations where item response theory based methods may not be usable, recent findings have suggested that the performance of the MHP and Logit needs to be examined in detail. The present research examined the impact of manipulating sample size, the ratio of focal to reference group members, and the number of levels of the matching criterion on the performance of the MHP and Logit. Reference and focal groups consisted of 7,320 majority group Scholastic Aptitude Examination (SAT) takers and 791 minority group SAT takers. Other samples of 1,000 reference and 500 focal group members were obtained through a random number generating program. Four, 8, and 12 levels of matching on the SAT score were used. Results suggest that the MHP and Logit are sensitive to these manipulations. As sample sizes decreased, mean values for the MHP and Logit decreased, and agreement between them declined. As the number of levels of matching increased in the full sample condition, agreement between Logit and the MHP also dropped. Four tables present study findings, and there is a 23-item list of references. (Author/SLD)
Descriptors: College Entrance Examinations, Comparative Analysis, Control Groups, Equations (Mathematics), Evaluation Methods, Experimental Groups, High School Students, High Schools, Item Bias, Item Response Theory, Mathematical Models, Minority Groups, Research Methodology, Research Problems, Sample Size, Sampling, Test Items
Publication Type: Reports - Descriptive; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Logit Analysis; Mantel Haenszel Procedure; Matching to Sample Procedure; Scholastic Aptitude Test
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, CA, April 1992).