Previous research has shown the effectiveness of peer reviewing on the improvement of writing quality. However, the fact that students themselves, arguably novices, judged the improvement leads to concerns about the validity of peer reviewing. We measured writing quality before and after peer reviewing using Coh-Metrix, which computationally evaluates the linguistic properties of writings. Participants also evaluated the quality of their peer writings using the SWoRD system. Both measurements, particularly the computational measurement, confirmed the effectiveness of peer reviewing. In addition, the computational measures found that awareness of cohesion, including the clarity, explicitness, and concreteness of writing, improved over the course of peer reviewing. The results are discussed, along with their possible implications for the complementary roles of peer reviewing and computational writing measurements.