ERIC Number: ED483777
Record Type: Non-Journal
Publication Date: 2004-Mar
A Study of Gender-Based and Ethnic-Based Differential Item Functioning (DIF) in the Spring 2003 Idaho Standards Achievement Tests Applying the Simultaneous Bias Test (SIBTEST) and the Mantel-Haenszel Chi Square Test: Idaho Standards Achievement Tests; Reading, Language Usage, and Mathematics, Grades 4, 8, and 10.
Stoneberg, Bert D.
Idaho State Department of Education
Differential item functioning (DIF) occurs when the responses of students with approximately equal ability differ systematically based on their group membership. Idaho contracted with the Northwest Evaluation Association (NWEA) to provide the Idaho Standards Achievement Tests of reading, language arts and mathematics at grades 4, 8 and 10. The tests included items in the NWEA Rasch-calibrated item bank and items written by Idaho teachers that were equated to the item bank metric. NWEA provided a random sample of data representing about half of the students taking part in the spring 2003 assessment. This study screened for gender-based (female-male) and ethnic-based (Hispanic-White) DIF. The Simultaneous Item Bias Test (SIBTEST) and the Mantel-Haenszel Chi Square Test were used to detect statistically significant DIF. The Bonferonni correction was employed to control for Type I error. Effect size procedures were used to determine whether statistical differences were large enough to have practical meaning. DIF findings from a NWEA analysis of spring 2003 test results were also noted. The proportion of items exhibiting gender-based DIF ranged from seven percent for the Grade 4 reading test to 37 percent for the Grade 10 mathematics test. The proportion of items exhibiting ethnic-based DIF ranged from seven percent for the Grade 8 mathematics test to 19 percent for the Grade 4 tests of reading and language arts. The study recommended that items exhibiting DIF be reviewed by curriculum specialists before further use, that the impact of the high occurrence of DIF on the ability to accurately interpret the spring 2003 student scores be examined, and that language requiring DIF analyses on all items used in future test administrations be added to the contract with the test developer.
Publication Type: Numerical/Quantitative Data; Reports - Evaluative
Education Level: N/A
Authoring Institution: Idaho State Dept. of Education, Boise.