The purpose of this study was to examine the effect of criteria-referenced formative assessment on achievement in the arts. Seventy-five schools in New York City were assigned to either the treatment or control condition. The treatment involved 3195 elementary, middle, or high school students instructed by 43 music, visual arts, theater, or dance teachers. The teachers were involved in a professional development program focused on formative assessment practices, particularly criteria-referenced peer and self-assessment. The control group consisted of 2445 students in classes instructed by 32 teachers who did not receive the professional development. Discipline-specific, performance-based pre- and post-measures were used to evaluate student learning. Fidelity of implementation was examined before the analysis of the treatment effect. Propensity score matching analysis was used to examine group differences in performance on the post-assessment. Results based on a sample of 611 matched pairs of students showed that, overall, criteria-referenced formative assessment had a statistically significant, positive effect (d = 0.26) on students' arts achievement.