The present study was designed to examine the relationship between performance on a standards-based assessment of reading comprehension in fourth grade and performance on a diagnostic battery of component skills of reading for a cohort of children who were followed from kindergarten through fourth grade. The findings demonstrate that the relationship between performance on component skills of reading and the proficiency classifications from the standards-based assessment is generally weak. They also demonstrate that the relationship between the proficiency classifications and a norm-referenced diagnostic measure of reading comprehension is only moderate. Furthermore, the study shows that the broad proficiency-level classifications of the standards-based assessment mask significant heterogeneity in children's performance on some of the component skills assessed, both within and across proficiency classifications. In turn, the ability of the standards-based assessment results to inform instructional interventions is low. The implications of these findings for instructional planning within standards-based accountability systems are discussed. (Contains 2 figures and 7 tables.)