Educational reform in the past decade has placed a very strong emphasis on teacher accountability and student achievement. This change in focus has caused a thorough reexamination of how teacher quality and effectiveness are defined in the field, and has triggered a reform of the teacher evaluation process. As a result, generic, all-encompassing evaluation systems are being developed to evaluate teachers across all subject areas, including physical education. This one-size-fits-all approach meets the more obvious goal of evaluation, which is to review and appraise the competence level of teachers, but questions arise about whether the evaluation will improve teacher effectiveness. This generic approach overlooks the actual context in which teaching takes place and disregards the subject matter being taught. If the primary goal of teacher evaluation is to help teachers become more effective, the evaluation system must provide teachers with objective, content-specific feedback to inform practice and guide them to improve their teaching effectiveness. This article discusses concerns related to using some of the more generic evaluation systems currently being implemented in schools to evaluate physical education teachers. It concludes with suggestions about how physical education teacher education programs can provide observation tools to their teacher candidates to improve teacher effectiveness.