Educators have realized the importance of engaging students in learning. Teachers often see participatory behaviors like "hand raising" as evidence of students being engaged in an activity. These indications of engagement do not capture motivational factors behind true engagement. A research team developed a five item scale to easily assess engagement while capturing psychological aspects of true engagement with the additional validity advantage of being grounded in Schlechty's theory of engagement. The goal of this current research effort was to extend the development of the checklist by estimating reliability of this informal measure. Results suggest a high degree of reliability between students' ratings across occasions. The end product of this study is a psychometrically sound, easy to use measure of true student engagement that may be used by educators at all grade levels and across disciplines. The scale has the additional advantage of providing data for teachers about the level of engagement in lesson deliveries across classes and lesson types. This information will promote more awareness and use of engaging strategies for teachers in practice.