Self-study of variations to task design offers a way of analysing how learning takes place. Over several years, variations were made to improve an assessment task completed by final-year teacher candidates in a primary mathematics teacher education subject. This article describes how alterations to a task informed on-going developments in self-study of one assessment task employed in an online subject. Analysis of my journal, notes from conversations with colleagues, teacher candidates' work on the task and responses to online forums, and survey data inspired variations focused on better exploration of key concepts involved in the task, raising of focal awareness, developing a stronger professional eye in the students and the author, adaptations for multiple curriculum levels, and explorations of dual teacher-student perspectives. The overall challenge has been to support teacher candidates to learn to design effective open-ended tasks with a critical professional eye. Descriptions of the changes made to the task and the development of my own professional eye as a consequence of the application of self-study are included. Data show that variations to the task increased teacher candidates' understanding of mathematics problem posing and generated pedagogical insights for task design.