ERIC Number: EJ1065345
Record Type: Journal
Publication Date: 2015
Abstractor: As Provided
Reference Count: N/A
Uncovering Multivariate Structure in Classroom Observations in the Presence of Rater Errors
McCaffrey, Daniel F.; Yuan, Kun; Savitsky, Terrance D.; Lockwood, J. R.; Edelen, Maria O.
Educational Measurement: Issues and Practice, v34 n2 p34-46 Sum 2015
We examine the factor structure of scores from the CLASS-S protocol obtained from observations of middle school classroom teaching. Factor analysis has been used to support both interpretations of scores from classroom observation protocols, like CLASS-S, and the theories about teaching that underlie them. However, classroom observations contain multiple sources of error, most predominantly rater errors. We demonstrate that errors in scores made by two raters on the same lesson have a factor structure that is distinct from the factor structure at the teacher level. Consequently, the "standard" approach of analyzing on teacher-level average dimension scores can yield incorrect inferences about the factor structure at the teacher level and possibly misleading evidence about the validity of scores and theories of teaching. We consider alternative hierarchical estimation approaches designed to prevent the contamination of estimated teacher-level factors. These alternative approaches find a teacher-level factor structure for CLASS-S that consists of strongly correlated support and classroom management factors. Our results have implications for future studies using factor analysis on classroom observation data to develop validity evidence and test theories of teaching and for practitioners who rely on the results of such studies to support their use and interpretation of the classroom observation scores.
Descriptors: Factor Structure, Multivariate Analysis, Scores, Factor Analysis, Classroom Observation Techniques, Middle School Teachers, Protocol Analysis, Error of Measurement, Interrater Reliability, Inferences, Validity
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: email@example.com; Web site: http://www.wiley.com/WileyCDA
Publication Type: Journal Articles; Reports - Research
Education Level: Middle Schools; Secondary Education; Junior High Schools
Authoring Institution: N/A