Implementation of research-based, Tier 1 behavior management strategies can be monitored to provide data-driven feedback and in support of integrity. The "Measure of Active Supervision and Interaction" (MASI) was developed to measure four behavior management practices (i.e., Praise, Correction, References to Behavior Expectations, Active Supervision) using systematic direct observation. This study was designed to address research questions related to reliability and validity by applying the MASI to evaluate staff behavior in seven out-of-school time programs. Findings indicate that two raters can complete the MASI with high agreement. Ratings are attributable largely to desirable sources of variance, and content validators positively rate the measure. Results are nonsignificantly correlated with established implementation measures for Positive Behavior Interventions and Supports.