The aims of the study were twofold: (a) to investigate how school size affects absentee rates in physical education (PE) and (b) to examine the experiences of students and teachers at a small rural high school in relation to attendance in PE at their school. The absentee rates in PE among all students (N = 6928 students) in a county in Norway were collected, and interviews with 15 students and two teachers at a small rural high school in the county were conducted. The results showed that the absentee rates in PE were significantly lower in small high schools (<200 students) located in rural areas, compared with larger-sized schools located in communities with more people. Small high schools also produced a lower percentage of students with an absentee rate above 10%. Furthermore, in contrast to small high schools, the absentee rate among girls was significantly higher than among boys in large high schools. Finally, the follow-up study showed that both students and teachers in a small rural high school reported a high level of relatedness, and both teachers and students claimed that this relatedness deterred absenteeism in PE. This finding may help explain the low absentee rates among students at small high schools.