The purposes of this article are to illumine the racist genealogy of gifted education policies and practices in the United States, to demonstrate how deficit discourses continue today, and to provide personal examples from the field of how educators can begin to question the status quo, resist taken-for-granted assumptions, and alternatively make substantive changes at the local level. I also aim to demonstrate how giftedness is an example of whiteness as property, or unearned white privilege, that, unintentionally or not, maintains a social caste system in schools.