This study used a stratified random sample of classes in Zurich, Switzerland, comprising approximately 2,000 students whose academic performances in math and language were assessed across primary and lower secondary education. Based on this longitudinal data, the study investigated the association of social inequalities with the baseline of, and gains in, academic performance. The study focused on growing social disparities in academic performance during compulsory education, taking into account disparities in the social backgrounds of students as well as in social deprivation of school attendance areas. The results of a multilevel growth curve analysis implemented to model academic performance development at student and school district levels suggest cumulative disadvantages for students from lower socioeconomic backgrounds. These students (a) start school with lower initial knowledge and (b) experience lower improvements in academic performance. The findings also suggest that more advantaged school attendance areas achieved higher average performance levels in the early stages of primary education despite controlling for student socioeconomic backgrounds. On average, however, this gap in academic performance between more advantaged and more deprived attendance areas did not appear to widen over the subsequent years of compulsory schooling.