ERIC Number: ED466712
Record Type: Non-Journal
Publication Date: 2002-Apr
Reference Count: N/A
Extending DIF Methods To Inform Aggregate Reports on Cognitive Skills.
Milewski, Glenn B.; Baron, Patricia A.
Maximum usefulness of information provided to teachers and program staff by the enhanced score reports of the Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) depends on the development of aggregate skill reports; that is, reports that provide skill information aggregated across groups of students, such as grade level within high school. In this study, differential item functioning (DIF) methodology is extended to individual skill performance in order to compare aggregate groups, like schools or states, to the total population matched on an overall score. The results of four DIF detection methods used for this purpose are compared. Mantel-Haenszel and standardization procedures were calculated using binary skill classification data, and polytomous standardization and analysis of covariance (ANCOVA) were calculated using continuous posterior skill mastery probabilities. Results indicate that the ANCOVA approach was very sensitive to differential skill performance between the aggregate group and the population matched on ability. The Mantel-Haenszel, standardization, and polytomous standardization approaches provided consistent results that were less sensitive to differential skill performance. Taken together, these findings suggest that the ANCOVA approach may be too sensitive to differences in skill performance across groups matched on ability. Limitations and directions for future research are discussed. An appendix provides additional information about student skills. (Contains 10 tables and 22 references.) (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Merit Scholarship Qualifying Test; Preliminary Scholastic Aptitude Test