Formative evaluation and its associated methodology of reflection on practice are used extensively in academic staff development. In reflecting on formative evaluation processes in both more traditional and newer programmes conducted at a university of technology, a number of variables reported in the literature were observed to have influenced academic staff members' ability to reflect and change practice. Drawing on illustrative cases, this paper argues that explicit attention needs to be given to additional variables concerned with the nature of the knowledge being taught, academic identity, and the availability of a community of educational practitioners if academic developers are to foster critical reflection as an essential element of formative evaluation and productive change in practice.