This article explores the importance of critical discourse in physical education (PE) that focuses on how understandings of ability are defined, practised, and potentially altered. Research continues to indicate that physical educators continue to draw on narrow notions of ability which are influenced by the presence of a pervasive performative culture. Traditional understandings of ability often fail to reflect the wider aims of PE such as developing young people's physical literacy. The theoretical concepts of Bourdieu have been used to explain processes that serve to reinforce "legitimate" notions of ability. The significance of the field of PE has been highlighted where habitus and capital inform understandings of ability in PE, and reinforce practices that privilege certain students. The data for this article are based on in-depth, semi-structured interviews with six PE teachers who were part of a yearlong study on young people's experiences of ability in secondary PE in England. Within interviews PE teachers defined ability in broad terms and differentiated between the purpose of PE and sport. In practice, teachers placed a distinct emphasis on defining ability in terms which privileged students who either had a reputation for sporting excellence or who demonstrated desired forms of sport-related physical capital in lessons. These discourses and practices were reinforced through individual habitus and through a sense of shared "mastery of the common code" among the teachers. The findings have implications for understanding the tensions within the field of PE that have evolved from previous, and continuing, debates on the distinction between PE and sport and understandings of the purpose of PE. In addition, they highlight the challenges that teachers can experience in aligning their wider views of ability with their "legitimate" ability-based practices and working towards supporting more equitable and inclusive PE experiences.