Participation in post-compulsory physics is a matter of longstanding concern from both economic and equity perspectives. In considering this issue, this study draws upon Bourdieu's theory of social practice, particularly notions of the 'cultural arbitrary', to explore what insights into post-compulsory physics choice might be provided by students who could have chosen physics, but did not, opting for other sciences instead. Utilising survey data from over 13,000 year 11 (ages 15/16) students in England, as well as qualitative interviews with 70 students of the same age, findings reinforce the key role of individual aspirations in subject choice. However, they also highlight the influence of the cultural arbitrary of physics (e.g. as difficult, masculine), which leads many students to conclude that physics is not 'for me' and hence choose other paths. This finding emphasises the entrenched nature of the challenges facing efforts to increase equity in post-compulsory physics participation.