NotesFAQContact Us
Search Tips
ERIC Number: ED566252
Record Type: Non-Journal
Publication Date: 2014-Mar
Pages: 18
Abstractor: ERIC
Implications of QRIS Design for the Distribution of Program Ratings and Linkages between Ratings and Observed Quality. OPRE Research Brief 2014-33
Tout, Kathryn; Chien, Nina; Rothenberg, Laura; Li, Weilin
Administration for Children & Families
This Brief compares three hypothetical Quality Rating and Improvement Systems (QRIS) that use different rating structures: block, points, and hybrid. Because the quality standards in the hypothetical QRIS are held relatively constant across structures, analyses can be conducted to determine how structure relates to key QRIS outcomes. Three outcomes are examined: the distribution of programs across ratings levels, the linkages of ratings with measures of observed quality, and the scores of individual quality components within each structure. Findings indicate that the distribution of ratings is significantly related to structure. Whereas fewer than one-fifth of programs achieved a Level 3 or 4 in the block structure, over 70% of programs achieved a Level 3 or 4 in the points and hybrid structures. Rating levels produced by each of the three structures were significantly correlated with observed quality as measured by the Early Childhood Environment Rating Scale--Revised (ECERS-R). However, the points structure was the only structure to produce quality levels in which observed quality was significantly different between each level. The points structure also captured the greatest range of ECERS-R scores with a 1.61 point spread between Level 1 and Level 4 compared to 0.13 and 1.14 point spreads for the block and hybrid structures respectively. Scores across rating levels in the rating structures showed different patterns for specific quality components, with some domains (Health and Safety, Assessment and Accreditation) scoring high regardless of level and structure, others (Family Partnerships) scoring relatively low and others (Teacher Qualifications and Director Qualifications) demonstrating how quality component scores can differ across structures. The analyses are limited in their application to QRIS because the data were collected with a unique sample and were not collected in the context of a "real" QRIS. Nevertheless, this Brief offers research evidence that can be useful to QRIS administrators as they weigh different design options and their potential consequences.
Administration for Children & Families. US Department of Health and Human Services, 370 L'Enfant Promenade SW, Washington, DC 20447. Web site:
Publication Type: Reports - Evaluative
Education Level: Early Childhood Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Administration for Children and Families (DHHS), Office of Planning, Research & Evaluation; Child Trends
Identifiers - Assessments and Surveys: Early Childhood Environment Rating Scale
Grant or Contract Numbers: HHSP23320095631WC
IES Cited: ED555393