In this paper, we use the work of philosopher Gilles Chatelet to rethink the gesture/diagram relationship and to explore the ways mathematical agency is constituted through it. We argue for a fundamental philosophical shift to better conceptualize the relationship between gesture and diagram, and suggest that such an approach might open up new ways of conceptualizing the very idea of mathematical embodiment. We draw on contemporary attempts to rethink embodiment, such as Rotman's work on a "material semiotics", Radford's work on "sensuous cognition", and Roth's work on "material phenomenology". After discussing this work and its intersections with that of Chatelet, we discuss data collected from a research experiment as a way to demonstrate the viability of this new theoretical framework.