This study describes the development of the Staff Implementation Observation Form, an instrument to assess staff competence delivering an intervention to youth in group home care with behavioral or emotional disorders. This instrument assesses staff skill at implementing the key treatment components, including building relationships with youth, teaching skills to youth, implementing a token economy, and establishing a youth self-government system. The pilot study, observing 92 staff members, found that the instrument possesses promising psychometric qualities, including adequate inter-rater agreement, internal scale consistency, and predictive validity. A cluster analysis identified groups of staff that had low, average, and high implementation. The instrument allows administrators to examine item-level implementation of residential staff to identify specific training needs within each subscale. (Contains 1 figure and 3 tables.)