There is no standard definition of empathy, but the concept is assumed to be innately pro-social and teachable regardless of factors such as power dynamics or other manifestations of social injustice within a society. Such assumptions in discursive practices, whether academic, popular, or pedagogical, obscure the emergence of two important questions: What does it mean when we cannot empathize with another? And could it be that we may gain greater insight from the examination of empathy's limits and failures than the hopes we have for its success? Through an exploration of some of Edith Stein's and Judith Butler's work on the subject, I propose that discussions of empathy, particularly in education, must be grounded in social context. Once this is done, assumptions about empathy must be continually troubled if one is to have a cogent conversation--whether as a philosopher, social theorist, educator, or policy maker--about what empathy is (or is not) and what it does (or does not) make possible.