Teachers' assessment enhances students' learning in many situations in physical education (PE). The present study aims to examine the issue of potential unequal assessment of students due to their different physical abilities in PE in upper secondary school, where the research question is: What experiences do students with high or low levels of physical fitness have of their PE teacher's assessment of and for learning? Individual interviews were conducted with thirteen boys and thirteen girls in the second year in upper secondary school to answer the question. Respondents were recruited on the basis of scores on physical fitness tests (measurement of oxygen uptake). Thirteen respondents scored on the lowest level and thirteen on the highest level of the fitness scale. Two main themes emerged from the analysis, revealing an unequal assessment of learning given to the students in the two groups, but a shared, equal experience of assessment for learning. Unfortunately, assessment for learning was experienced 'rarely or never', and the unequal assessment of learning did not favour the students with low physical fitness, who perhaps need more assessment than the students with high physical fitness. The study contributes new insight into students' experience of assessment due to unequal conditions for learning. The analysis revealed elements that were lacking in the PE teaching and assessment, especially for the students with low physical fitness. Another finding is that few students experienced that they owned their learning process, even though some students experienced to be in a learning environment, wherein the teacher gave them learning-promoting assessment. The study reinforces the finding in earlier research to integrate assessment for learning in theories of teaching in PE, and from this perspective we support further exploration of assessment for learning as well as the concept of assessment literacy.