Classroom observations have been increasingly used for teacher evaluations, and it is important to examine the measurement quality and the use of observation ratings. When a teacher is observed in multiple classrooms, his or her observation ratings may vary across classrooms. In that case, using ratings from one classroom per teacher may not be adequate to represent a teacher's instructional quality. Drawing on the Measures of Effective Teaching (MET) dataset, this study examined the variation of a teacher's classroom observation ratings across his or her multiple classrooms. The results indicate that the math classrooms accounted for 4.9 to 14.7% of the variance in the classroom observation ratings and English language arts (ELA) classrooms accounted for 6.7 to 15.5% of the variance in the ratings. The results of this study suggest that teachers' multiple classrooms should be taken into consideration when classroom observation ratings are used to evaluate teachers in high-stakes settings.