There are situations within middle school settings where measurements of students and teachers are used for high-stakes decisions. For example, student performance is used as an indicator of teacher quality or determines student eligibility for particular types of support services. Given the high-stakes nature of these types of assessments, understanding the strengths and limitations of the measurement is essential. Generalizability theory is a method to help better understand reliability and can be used to inform and improve measurement issues in middle school settings. The purpose of this article is to create a more informed consumer of measurement issues in middle school settings by focusing on the reliability of a particular measurement scenario: observation of middle school teacher behaviors. Teacher observations might be conducted by other teachers, principals, or other administrators or researchers as one indicator of instructional quality. We describe how a 1-facet crossed design and 2-facet crossed design can be applied to different observational protocols and offer considerations for those interested in conducting a generalizability study. We emphasize that no single design will apply to all generalizability studies and findings for a particular observational measure will not necessarily generalize to other observational measures. Instead, the researcher must consider the purpose and use for their particular observational measure.