ERIC Number: ED344937
Record Type: RIE
Publication Date: 1992-Apr
Reference Count: 0
An Investigation of the Relationship between Reliability, Power, and the Type I Error Rate of the Mantel-Haenszel and Simultaneous Item Bias Detection Procedures.
Ackerman, Terry A.; Evans, John A.
The relationship between levels of reliability and the power of two bias and differential item functioning (DIF) detection methods is examined. Both methods, the Mantel-Haenszel (MH) procedure of P. W. Holland and D. T. Thayer (1988) and the Simultaneous Item Bias (SIB) procedure of R. Shealy and W. Stout (1991), use examinees' raw scores as a conditioning variable in the computation of differential performance between two groups of interest. As a result, the extent to which examinees' observed scores accurately reflect their true abilities plays an important role. If examinees are misrepresented by their observed score (as for a test with low reliability) then the ability of bias detection methods to determine item bias may not be very accurate. Results of Monte Carlo studies (40-item test, 720 testing conditions) suggest that for a fixed-length test, the power of both statistics increases moderately as reliability is increased and substantially as sample size is increased. However, the combination of small sample sizes and high reliability results in a decrease of power. For most of the simulated conditions, the MH and SIB procedures have very similar rates of correctly rejecting the biased item. Sixteen plots illustrate the discussion. There is a 15-item list of references. (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Ability Estimates; Mantel Haenszel Procedure; Simultaneous Item Bias Procedure; Type I Errors
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, CA, April 21-23, 1992).