The purpose of this study was to analyze how the challenges of urban schools influence physical education teachers' emotional understanding and connections with their students and the implications on their teaching. Sixty-one elementary physical educators from an urban school district in the midwestern U.S. were interviewed multiple times (N = 136) over 3 years using interpretive methodology. Teachers reported five unique challenges that significantly shaped their thinking about students and their careers, along with strategies they used to overcome or manage those challenges. The challenges were: (a) insufficient instructional resources, (b) implementing culturally relevant pedagogy, (c) dealing with community violence, (d) integrating more games in curricula, and (e) teaching in a culture of basketball. Implications centered on the guilt-inducing nature of urban teaching, developing an informed and realistic vision of urban physical education, and the role of teacher preparation and professional development.