The Primary Teacher Education (PrimTEd) project was established in response to concerns about the pre-service preparation of primary teachers in South Africa. In order to inform the development of appropriate pre-service mathematics courses, an initial need in the PrimTEd project was to establish the nature of the mathematical knowledge of pre-service teachers both near the start and also near the end of their studies, through an online assessment. This paper describes the design of this PrimTEd online mathematics test and reports on the performance on the test of first-year and fourth-year primary mathematics student teachers, at three different universities. The overall performance of students together with the small differences between first and fourth year student performance indicate a need for student teachers to revisit primary school mathematics in a way that provides a deep understanding of key mathematical concepts in order to be better prepared for future teaching careers.