Adapted from the Early Childhood Environment Rating Scale-Revised, the Chinese Early Childhood Program Rating Scale (CECPRS) is a culturally comparable measure for assessing the quality of early childhood education and care programs in the Chinese cultural/social contexts. In this study, 176 kindergarten classrooms were rated with CECPRS on eight dimensions of program quality. Multivariate generalizability theory (MG-theory) was applied to evaluate the reliability of CECPRS ratings. Empirical findings indicate that CECPRS ratings showed excellent reliability, both on the subscales and for the composite. Based on the findings of the D-study for different measurement protocol scenarios, for achieving a good balance between reliability and cost/efficiency, we recommend using two independent raters for each kindergarten classroom. Furthermore, the application of MG-theory in this context broke new ground in the assessment of child care program quality, as the MG-theory provides research and practical possibilities not readily available under the traditional reliability methods.