The current educational reform movement in the United States is focused on measuring the effectiveness of teachers. One component of teacher effectiveness is student achievement. The effectiveness of using PE Metrics as a measure of student achievement in a physical activity setting with a low socioeconomic, culturally diverse population was examined in this study. Two raters scored second and fifth grade children (N = 90) on skills using the PE Metrics system. Skills assessed included skipping, galloping, dribble with hand jog, basketball, soccer, and overhand throw. An item analysis showed that teachers could use the majority of the items to discriminate skill level. Strengths and weaknesses of the PE Metrics system were discussed.