NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED489949
Record Type: Non-Journal
Publication Date: 2004-Mar
Pages: 74
Abstractor: Author
ISBN: N/A
ISSN: N/A
EISSN: N/A
A Study of Gender-Based and Ethnic-Based Differential Item Functioning (DIF) in the Spring 2003 Idaho Standards Achievement Tests Applying the Simultaneous Bias Test (SIBTEST) and the Mantel-Haenszel Chi Square Test
Stoneberg, Bert D., Jr.
Online Submission
The purpose of this study was to screen for gender-based (female-male) and ethnic-based (Hispanic-White) differential item functioning (DIF) in the 2003 Idaho Standards Achievement Tests of reading, language arts and mathematics at grades 4, 8 and 10. The vendor, the Northwest Evaluation Association (NWEA), assembled the computer-administered tests using items from its Rasch-calibrated item bank and items from Idaho teachers that were equated to the item bank metric. The absence of DIF statistics for any of these test items led to this study. NWEA provided random samples of data representing about half of the students completing each grade-level test that ranged from 8,659 for the tenth grade mathematics test to 9,676 for the fourth grade language usage test. This study used the Simultaneous Item Bias Test (SIBTEST), which assesses item dimensionality, and the Mantel-Haenszel Chi Square Test to detect statistically significant DIF in the individual items. A Bonferonni correction controlled for Type I error. Effect size procedures described whether the observed statistical differences were large enough to have practical meaning. This study also noted DIF findings from a concurrent NWEA analysis of item parameter invariance in the 2003 Idaho tests. The proportion of items exhibiting moderate to high gender-based DIF ranged from seven percent for the fourth grade reading test to 37 percent for the tenth grade mathematics test. The proportion of items exhibiting moderate to high ethnic-based DIF ranged from seven percent for the eighth grade mathematics test to 19 percent for the fourth grade tests of reading and language arts. The study recommended that NWEA and independent curriculum specialists review all items exhibiting moderate to high DIF before further use, that the state examine the impact of the high occurrence of DIF on the ability to accurately interpret the spring 2003 student scores, and that the state add language to the vendor contract requiring DIF analyses of all items used in future administrations of the statewide tests. (Contains 6 tables, 1 figure, and 3 appendices.)
Publication Type: Reports - Evaluative
Education Level: Grade 10; Grade 4; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Idaho
Grant or Contract Numbers: N/A