In 1997, the Ontario government, like many other jurisdictions, undertook systemic reform of their elementary school mathematics programme, developing a new mathematics curriculum, report card, and province-wide assessment. The curricular reform embodied a new vision of mathematics learning and instruction that emphasized instruction using challenging problems, the student construction of multiple solution methods, and mathematical communication and defence of ideas. While the design of the original large-scale assessment incorporated much of the latest research and theory on effective practices at that time, these traditional item development and scoring practices no longer adequately assess mathematics achievement in reform-inspired classrooms. The difficulties of marrying traditional assessment practices with a reform-inspired curriculum could be addressed by creating a construct definition from the recent research findings on students' mathematical development in reform-inspired classrooms. The importance, challenges and implications of redefining the construct on the basis of existing research on students' mathematical development, as well as collapsing the traditional content-by-process matrix for item development, are explored. (Contains 8 figures and 3 notes.)