Despite policy efforts to encourage multiple measures of performance in newly developing teacher evaluation systems, practical constraints often result in evaluations based predominantly on formal classroom observations. Yet there is limited knowledge of how these observational measures relate to student achievement. This article leverages the random assignment of teachers to classrooms from the Measures of Effective Teaching (MET) study to identify teacher effectiveness using scores from the Framework for Teaching (FFT) instrument, one of the most widely used classroom observation protocols. While our evidence suggests that teacher performance, as measured by the FFT, is correlated with student achievement, noncompliance with randomization and the modest year-to-year correlation of a teacher's FFT scores constrain our ability to causally identify effective teachers. Implications for policy and practice are discussed.