NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1109923
Record Type: Journal
Publication Date: 2013-Sep
Pages: 29
Abstractor: As Provided
ISSN: EISSN-2330-8516
An Investigation of the Efficacy of Criterion Refinement Procedures in Mantel-Haenszel DIF Analysis. Research Report. ETS RR-13-16
Zwick, Rebecca; Ye, Lei; Isham, Steven
ETS Research Report Series, Sep 2013
Differential item functioning (DIF) analysis is a key component in the evaluation of the fairness and validity of educational tests. Although it is often assumed that refinement of the matching criterion always provides more accurate DIF results, the actual situation proves to be more complex. To explore the effectiveness of refinement, we conducted a simulation study consisting of 40 conditions that varied in terms of amount and pattern of DIF, sample sizes, and ability distributions. We found that the effectiveness of refinement was heavily dependent on whether DIF was balanced (with positive DIF values compensating for negative DIF values) or unbalanced (all in one direction). In balanced conditions, the unrefined method generally produced better results, whereas, in unbalanced conditions, the opposite was true. In the absence of information about the pattern of DIF, it is probably best to choose the refined method because it is only slightly disadvantageous in balanced conditions, whereas the unrefined method can be substantially disadvantageous in certain unbalanced conditions.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A