In the majority of states using Quality Rating and Improvement Systems (QRIS) to improve children's school readiness, the Early Childhood Environmental Rating Scale-Revised (ECERS-R) is a core assessment of preschool program quality and is central to QRIS metrics and incentive structures. The present study utilizes nationally representative data from the Early Childhood Longitudinal Study-Birth Cohort to examine relations between the ECERS-R and children's academic, language, and socioemotional functioning at age five years. After using a rich set of controls, we found little evidence that the ECERS-R related to children's development. Further, higher levels of quality failed to improve growth in academic, language, or socioemotional skills and behaviors for children with more exposure to sociodemographic risk. Implications of these findings are discussed with regard to recent policy initiatives and strengthening the measurement of quality in early childhood education settings.