The importance of the role of language/discourse in the learning and teaching of mathematics is noted in many mathematics curricula and standards documents. In the research literature, this role has been widely theorised from a Vygotskian perspective. This perspective is limited by some of its underlying assumptions, including an instrumental and systemic view of language as tool and its basis in dialectic. In this paper, I propose a Bakhtinian, dialogic perspective as an alternative. I focus my discussion on the long-standing issue of the relationship between formal and informal mathematical language in the learning and teaching of mathematics. I illustrate this discussion with an examination of interaction in an elementary school mathematics classroom in Québec, Canada. Based on Bakhtin's ideas, I argue that mathematical meaning emerges through locally produced, situated dialogic relations between multiple discourses, voices and languages in mathematics classroom interaction. From this perspective, students do not follow a linear path from informal to formal mathematical discourse; rather, they work with the teacher to expand the repertoire of possible ways to make meaning in mathematics.