Meta-analysis can provide a robust description of the impact of educational reforms and also offer an opportunity to explore the conditions where such reforms are more or less effective. This article describes a meta-analysis on the impact of cooperative learning on students' chemistry understanding. Modifiers in the meta-analysis are purposefully chosen to model instructors' decisions in implementing cooperative learning. Modifiers investigated include: using cooperative learning periodically or in every class period; setting a maximum group size at four or smaller versus five or larger; using closed-ended or open-ended assessments; and assessing a single topic or assessing the cumulative topics in the course. The results showed cooperative learning's effectiveness is robust across a wide range of instructional decisions except no evidence of effectiveness was found with cumulative assessments. The overall results from the meta-analysis provide a benchmark for evaluating future efforts to evaluate pedagogical interventions in chemistry.