ERIC Number: ED353308
Record Type: Non-Journal
Publication Date: 1992
Reference Count: N/A
Comparison of Empirical and Judgmental Methods for Detecting Differential Item Functioning.
Hambleton, Ronald K.; Jones, Russell W.
The purpose of this study was to improve both statistical and judgmental methods for detecting potentially biased test items in an attempt to examine the agreement between the results obtained with these methods. If greater agreement between methods can be achieved, test items can be more effectively screened using judgmental methods prior to field testing or actual test administrations. Several methodological shortcomings of current empirical and judgmental methods were addressed. The test data came from samples of 2,000 Native Americans and 2,000 Anglo-Americans who took a 4-choice 150-item Statewide Proficiency Test. Fifteen Native American educators provided item bias reviews. To reduce computer time and facilitate the analyses, 75 items on the test were analyzed. The Mantel-Haenszel procedure was applied to the item responses for the two subject groups. The results suggest that s somewhat higher level of agreement between methods was obtained than has been observed in other studies. The use of cross-validation in empirically identifying potentially biased items was one reason for the higher level of agreement. However, the judgmental method implemented in this study appeared to have several shortcomings. Practical implications of the findings are presented. Two tables and one figure are included. (Author/RLC)
Descriptors: Achievement Tests, American Indians, Anglo Americans, Comparative Analysis, Item Bias, Judgment Analysis Technique, Multiple Choice Tests, Occupational Tests, Racial Differences, Reliability, Research Methodology, Research Problems, State Programs, Statistical Analysis, Test Results, Testing Programs
Publication Type: Reports - Research; Speeches/Meeting Papers; Tests/Questionnaires
Education Level: N/A
Authoring Institution: N/A