In a search for ways to capture foundation phase children's competence in mathematics, a small research team was put together at a South African university four years ago. At first, working only in a single school where the objective was to model 80 learners' growth in competence over four years, the team transformed into a very different entity. This article narrates the founding and development of a community of research practice, which eventually included undergraduate- and postgraduate students, researchers from institutions in Germany, Switzerland and South Africa, teachers at local schools close to the university, and an educational research and survey company. Supported by funding from four different sources, the project is fairly close to reaching the goal of delivering a standardised mathematics competence test for 4-8 year-olds, which is a first of its kind for the country where educational measurement seems to be at a crossroads. In the story line of this community of practice, it is evident that progress can be assessed best by the learning that has been taking place in a community with a notable diversity of people and interests, but with the shared goal of investigating children's mathematical behaviour on a measure that can be trusted.