This paper advocates categorization of engagement on the basis of a logical relationship with the outcomes of a study and the use of student attributes that are logically related to engagement and/or achievement as covariables. Results from a study involving nine engagement categories, measures of formal reasoning ability, locus of control and science achievement are used to highlight issues related to student engagement measures. The possibility of increasing the reliability of selected engagement categories through manipulation of student behavior is discussed. Generalizability theory is used to plan reliable data collection procedures and to assess the reliability of engagement measures. (Author)