As the interest in improving children's early math skills has grown, due in part to their strong associations with later overall school achievement, questions have been raised regarding teachers' knowledge about children's math abilities. The current study uses hierarchical linear modeling to examine the associations between teachers' judgments of children's math skills using an indirect rating scale assessment and children's performance on two direct assessments of their number sense and geometry and measurement skills. Approximately 40% of the variance in the rating scale completed by the teacher is attributable to systematic differences between classrooms, not specifically to the child. Overall, the association between teacher report and students' skills is approximately r = 0.50, which suggests that teachers can accurately determine whether students are above or below the mean but do not appropriately rate students as high or low as a direct assessment of their skills would indicate. This finding has implications for teachers, particularly in terms of teacher preparation, and for researchers, because of the information regarding the accuracy of teacher rating scales of preschool students' math skills. (Contains 3 tables.)