Schools, districts and inspectorates routinely use non-specialists to observe lessons for accountability and professional development purposes. However, there is little empirical research on how well non-specialists observe lessons. We describe two pilot studies in which education professionals made judgements about mathematics lesson observation reports, written by both specialists and non-specialists. In terms of providing feedback to the observed teachers, the professionals considered the specialists' reports to be significantly more useful than the non-specialists' reports. Written advice about a teacher's practice influenced these judgements. The paper considers theoretical and practical implications, as well as limitations of our findings. [For the complete proceedings, see ED597799.]