To track the quality of instruction delivered at the University of Western Australia, the university surveys all units using its Students' Unit Reflective Feedback (SURF) metrics, and faculties use these metrics to benchmark student satisfaction. Consequently, teaching staff are actively encouraged to adopt teaching practices that will increase the average levels of these metrics. Using a 'before-after' approach, we compared these metrics before and after the implementation of improved teaching practices that addressed specific weaknesses identified through student responses for two undergraduate units. Despite the implementation of improved teaching practices, SURF scores did not increase significantly for the two units assessed, due possibly to the components of the academic programme covered by the SURF questions, the field-based nature of the units, and the difficulty in measuring an increase in the SURF scores of units which already score highly. This poor sensitivity of the SURF metrics to the implementation of improved teaching practices could have implications for their use for performance assessment by the university.