Direct behavior ratings have been identified as a practical and feasible alternative to direct observation of behavior for monitoring behavioral progress. Despite the evidence of usability, there have been calls for further examination of direct behavior ratings using different behaviors and scales. To this end, we examined the ratings of schoolwide behavioral expectations by teachers and outside observers, as well as direct observation of academic engagement by outside observers in upper elementary school language arts classrooms. Across sessions, correlations between raters (range r = 0.82-0.91) and between measures (range r = 0.64-0.98) were moderate to high; significant differences between raters existed when rating students identified with negative behavior (n = 12) as opposed to positive behavior (n = 12). Findings are discussed in terms of implications for research and practice, limitations of the study, and the necessity for further research and development.