In view of the current emphasis on teachers' contextual assessment of emergent literacy skills, we need to determine the degree to which such assessments are valid judgments about children's early literacy skill development. One approach to this issue would be to examine closely the relationship between teacher ratings of literacy skills and student performance on individually-administered standardized measures with prior evidence of construct validity. In this study, we examine this issue by investigating the discrepancy between assessment information derived from teacher ratings and from formal tests, and by analyzing the variables that influence teacher judgments and their contextual assessments of kindergarten students' emergent literacy skills. In this study, we compared outcomes derived from teacher ratings of emergent literacy skills with those obtained on standardized direct measures with multiple sources of validity evidence. We collaborated with teachers in nine different schools to assess the emergent literacy skills of 205 kindergarten students using the two assessment measures: the Teacher Rating Scale, and the Word Reading subtest of the Wechsler Individual Achievement Test. Findings reveal systematic discrepancy between kindergarten teachers' ratings of students' emergent literacy skills and students' performance on a standardized direct measure with prior evidence of construct validity. Although the measures assess the same constructs, teachers' ratings of emergent literacy skills are severe in comparison with results derived from direct assessment. Results suggest that the use of teacher rating scales and checklists in early literacy assessment may have both positive and negative implications. On the positive side, there is a moderately strong positive correlation between the teacher rating scale of literacy skills, the TRS, and a psychometrically sound measure of emergent literacy with prior validity evidence, the WIAT-II Word Reading subtest. On the negative side, TRS ratings are affected by child and family characteristics despite best efforts to circumvent bias and diminish measurement error. Our findings, and those of other researchers, point to the need for further study of the influence of extraneous variables on teachers' contextual assessment of emergent literacy skills. (Contains 1 note.)