The role of physical literacy within physical education (PE) has become a widely debated topic in recent years. Its role in educating children about physicality through embodiment, skill acquisition and reading the environment is argued to be of great benefit to children. However, whether children understand the role of PE in the development of these competencies is not clear, and this is even truer for children who have special educational needs (SEN). Drawing on qualitative phenomenological data from 30 children in key stages two and three (7 to 14 years of age) who have SEN, this article explores notions of physical fitness and physical literacy as understood by children in PE lessons. It aims to gain insight into the ways that children understand the purpose of PE, and places these perceptions within a physical literacy framework, using the National Curriculum for PE (NCPE) as a foundation. Findings demonstrate that children with SEN perceive PE as a means for improving physical fitness, whereas concepts surrounding physical literacy appear to be lost. The article concludes by making recommendations for factoring physical literacy components more forcibly into the PE curriculum, and through initial teacher training and continued professional development. (Contains 2 tables.)