NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ930637
Record Type: Journal
Publication Date: 2010
Pages: 10
Abstractor: ERIC
Reference Count: 14
ISSN: ISSN-0017-8055
Misrepresentations in Unfair Treatment by Santelices and Wilson
Dorans, Neil J.
Harvard Educational Review, v80 n3 p404-413 Fall 2010
In his 2003 article in the "Harvard Educational Review" (HER), Freedle claimed that the SAT was both culturally and statistically biased and proposed a solution to ameliorate this bias. The author argued (Dorans, 2004a) that these claims were based on serious computational errors. In particular, he focused on how Freedle's table 2 was constructed. He contended that the numbers in this critical table did not represent what Freedle claimed. When he reconstructed the table correctly, the effects that Freedle reported were reduced substantially in magnitude to the point where they did not warrant the corrective action that he proposed. Santelices and Wilson (2010) claim to have confirmed Freedle's results by replicating correlations between differential item functioning (DIF) and item difficulty across very highly related DIF indices. Santelices and Wilson concluded by suggesting that the profession had dismissed Freedle's findings because of methodological criticisms by Educational Testing Service (ETS) researchers, including the author, and that their study has dealt with these concerns. In Dorans (2004a), however, the author made only one passing reference to the correlation between DIF and difficulty in the entire article. In this commentary, the author demonstrates that Santelices and Wilson fail to address his technical criticisms and fail to provide evidence of DIF on the test editions studied. Instead of addressing his primary concerns, the article misattributes to him the easily refutable hypothesis that choice of a DIF index appreciably affects the correlation between DIF and item difficulty. In section 1, the author restates his major concerns with Freedle's calculations and his R-score. In section 2, he cites misrepresentations in Santelices and Wilson. In section 3, he questions whether the DIF/difficulty correlation is relevant to Santelices and Wilson's call to "investigate the cause behind DIF between African American and White SAT test-takers." It is the magnitude of DIF on an item that matters, not the correlation between DIF and difficulty. In section 4, he raises confirmation bias as a possible explanation for the design and execution of the Santelices and Wilson study. (Contains 2 notes.)
Harvard Education Publishing Group. 8 Story Street First Floor, Cambridge, MA 02138. Tel: 617-495-3432; Fax: 617-496-3584; e-mail:; Web site:
Publication Type: Journal Articles; Opinion Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: SAT (College Admission Test)