ERIC Number: ED339746
Record Type: RIE
Publication Date: 1991-Nov
Reference Count: 0
Screening Items for Bias: An Empirical Comparison of the Performance of Three Indices in Small Samples of Examinees.
Kromrey, Jeffrey D.; Parshall, Cynthia G.
A Monte Carlo study was conducted to compare the performance of three statistical indices of test item bias in small samples of examinees. The statistical indices compared were the Delta method, the Mantel-Haenszel (MH) method, and the Standardization method. Sample sizes of 50, 100, and 200 were examined. One thousand samples of each size were drawn with replacement from each of three archival data files from three teacher subject area tests (in the areas of elementary education, early childhood education, and specific learning disabilities). Each sample was drawn so that 80% of the examinees were sampled from a reference group and 20% were sampled from a focal group. Item bias was experimentally controlled in the study, and the effectiveness of the indices was evaluated as the proportion of such biased items appropriately identified. Previous research suggesting that item bias indices such as the MH and Standardization methods should only be applied to large samples may have been overly conservative. Results support the use of statistical screening for item bias, even with samples as small as 50 examinees, and with only 10 focal group members in each sample. The MH is the best performer of these three indices, although both the MH and Standardization methods are preferable to the Delta method. Three tables present simulation data. An 11-item list of references and 3 tables are included. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: University of South Florida, Tampa. Inst. for Instructional Research and Practice.; Florida State Dept. of Education, Tallahassee.
Authoring Institution: N/A
Identifiers: Delta Method; Empirical Research; Mantel Haenszel Procedure; Standardization; Subject Content Knowledge
Note: Paper presented at the Annual Meeting of the Florida Educational Research Association (Clearwater, FL, November 13-16, 1991).