Academic language has been shown to significantly contribute to success across school subjects. However, to date, there are no empirical studies addressing its development across primary school age. The present study investigated the growth of academic vocabulary and influential conditions from Grades 2 to 4 based on a newly developed and validated test instrument. Drawing on a German sample of 472 children (N = 281 language minority learners) we found evidence of a "Matthew effect" with the group of language minority learners showing both lower initial level and lower growth rates compared to their monolingual peers. In addition, results of simple and full conditional growth curve models underline the significant impact of family background (parental education, books at home) and of children's nonverbal cognitive abilities on the intercept as well as the growth of academic vocabulary, thereby supporting the assumption of an additional Matthew effect according to family background, particularly in monolingual majority language children. Our results indicate the need for an early promotion of the register of academic language in monolingual as well as language minority students.