Many aspects of higher education must be reconceptualised for massive open online courses (MOOCs). Formative and summative assessment of qualitative work in particular requires novel approaches to cope with the numbers involved. Peer review has been proposed as one solution, and has been widely adopted by major MOOC providers, but there is currently little evidence about whether it is appropriate or under what conditions. Here, we examine student participation, performance and opinions of a peer review task in a biomedical science MOOC. We evaluate data from approximately 200 student topic summaries and 300 qualitative peer reviews of those summaries, and compare these to student demographic data (gender, age, employment status, education, national language) and to performance in multiple choice tests. We show that higher performance in the written topic summary correlated with both higher participation in the peer review task and with writing higher quality peer reviews. Qualitative analysis of student comments revealed that student opinion on the usefulness of the peer review task was mixed: some strongly believed it benefitted their learning, while others did not find it useful or did not participate. We suggest instructional design strategies to improve student participation and increase learning gain from peer review in the MOOC context.