This paper describes lessons learnt whilst using an online peer review system in an undergraduate unit for pre-service teachers. In this unit, students learn to use information technologies as part of their future teaching practice. The unit aims to foster graduates who become lifelong reflective educators by providing opportunities to explore and reflect on how they might use technology in authentic learning situations. Whilst peer review is an appropriate activity for supporting critical thinking and reflective practice in this kind of unit, it requires a number of decisions to be made in relation to student preparation and support, implementation strategy, and technological infrastructure to make it work in specific contexts. Much research has been conducted in recent years to inform educators in making these decisions. However, there are still gaps in the research, particularly in how to improve the quality and consistency of feedback that students give to each other in their feedback. This paper describes the experiences of implementing an online peer review system aiming to improve quality and consistency of feedback. This exploration has revealed that we can learn much about ways to improve our teaching practices by giving students an opportunity to review each other's work and give feedback.