In the context of intensifying accountability for academic work internationally, there are increasing pressures on individual supervisors, departments and universities to evaluate the quality of doctoral supervision. Existing evaluation tools are focused at departmental rather than individual level and are mostly quantitative. Evaluation for supervisors is usually limited to reflective self-assessment and peer critique. It has been difficult to elicit sustained feedback from doctoral students regarding their experiences of working with an individual supervisor, or even a supervisory panel, for ethical and practical reasons. This article discusses one attempt to address this problem: an online survey instrument that draws on previous research and qualitative methods to elicit sustained reflective commentary from doctoral students, in an anonymous format, about their experiences of supervision. The article describes the design and development of the survey, named the Research Student Feedback Survey (RSFS), and raises a number of critical conceptual and methodological issues for further exploration and debate. (Contains 1 note.)