This paper investigates the reliability of the Florida Performance Measurement Systems' Summative Observation instrument. Developed for the Florida Beginning Teacher Evaluation Program, it provides behavioral ratings for teachers in a classroom setting. Data came from ratings of videotapes of nine teachers conducting actual lessons by nine teams of trained observers. Analysis of variance produced three estimates of reliability for each scale and subscale: (1) discriminant (across teachers); (2) stability (over time); and (3) interrater (among raters). Results indicate that the instrument appears sufficiently reliable to conduct classroom observations if ratings by at least two different observers are averaged to produce scores. Effective (positive) indicators of teacher behavior appear to be more reliably observed than ineffective (negative) indicators. Two domains--Management of Student Conduct, and Communication: Verbal and Nonverbal--appear too intercorrelated with the other domains for discrete reliable estimation of specific behaviors. Future research on this instrument should include validation, rater certification, norming and frame factors. Appendices contain: (1) background information on the knowledge base and the Florida teacher competencies; (2) indicators of the summative instrument; and (3) computation of reliability estimates. (BS)