Most research on mathematical play and learning is focused on early childhood. This study examines how mathematical play and learning manifest in older children in a mathematical videogame designed by the first author, "Rolly's Adventure." We examined how players experienced mathematical play as they played "Rolly's Adventure," with a particular focus on failure paired with feedback. We used video and audio recordings of the players and their bodies, and screen capture of their gameplay. Spoken language, physical gestures, and digital actions were our primary sources of identifying, understanding, and triangulating mathematical play. We found that players pass through five zones of mathematical play that build upon each other and closely interrelate, and that these zones each involve different types of failure, feedback, and learning experiences. This paper provides a productive definition of mathematical play, introduces a framework that describes players' mathematical play experiences, and presents five design principles that can be leveraged to support mathematical play.