ERIC Number: EJ1221162
Record Type: Journal
Publication Date: 2019-Aug
Abstractor: As Provided
Making Inferences about Teacher Observation Scores over Time
Briggs, Derek C.; Alzen, Jessica L.
Educational and Psychological Measurement, v79 n4 p636-664 Aug 2019
Observation protocol scores are commonly used as status measures to support inferences about teacher practices. When multiple observations are collected for the same teacher over the course of a year, some portion of a teacher's score on each occasion may be attributable to the rater, lesson, and the time of year of the observation. All three of these are facets that can threaten the generalizability of teacher scores, but the role of time is easiest to overlook. A generalizability theory framework is used in this study to illustrate the concept of a hidden facet of measurement. When there are many temporally spaced observation occasions, it may be possible to support inferences about the growth in teaching practices over time as an alternative (or complement) to making inferences about status at a single point in time. This study uses longitudinal observation scores from the Measures of Effective Teaching project to estimate the reliability of teacher-level growth parameters for designs that vary in the number and spacing of observation occasions over a 2-year span. On the basis of a subsample of teachers scored using the Danielson Framework for Teaching, we show that at least eight observations over 2 years are needed before it would be possible to make distinctions in growth with a reliability coefficients of 0.39.
Descriptors: Observation, Inferences, Generalizability Theory, Scores, Time, Longitudinal Studies, Reliability, Teacher Evaluation, Teacher Effectiveness, Classroom Observation Techniques, Elementary School Teachers, Secondary School Teachers
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: firstname.lastname@example.org; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Secondary Education; Elementary Secondary Education
Authoring Institution: N/A
Identifiers - Location: North Carolina (Charlotte); Texas (Dallas); Colorado (Denver); Florida; New York (New York); Tennessee (Memphis)