NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED562820
Record Type: Non-Journal
Publication Date: 2014
Pages: 10
Abstractor: ERIC
Reference Count: 7
Observer Use of Standardized Observation Protocols in Consequential Observation Systems
Bell, Courtney A.; Yi, Qi; Jones, Nathan D.; Lewis, Jennifer M.; McLeod, Monica; Liu, Shuangshuang
Society for Research on Educational Effectiveness
Evidence from a handful of large-scale studies suggests that although observers can be trained to score reliably using observation protocols, there are concerns related to initial training and calibration activities designed to keep observers scoring accurately over time (e.g., Bell, et al, 2012; BMGF, 2012). Studies offer little insight into how educational practitioners understand and score observation protocols. This lack of clarity on the factors that facilitate and constrain educators' learning and use of observation systems makes it difficult to implement training and quality control processes at scale. In order to effectively train administrators at scale, it is critical to understand how administrators learn to complete two major tasks--learning to create accurate scores and learning to have conversations around those scores that support instructional improvement. This study takes the perspective that scoring observations of classroom interactions is a complex socio-cognitive process that must be understood in order to improve observer training, and ultimately, score quality. This study suggests administrators have a great deal of knowledge they bring to bear on the observation process. They also bring a commitment to improving instruction to their observation work. In other words, they are not blank slates as they go through observer training. That said, certification data suggest observers have much to learn about how to accurately score lessons according to the protocol. It is unclear whether the knowledge and commitments observers bring is supportive of high quality scores or the improvement of instruction. Future studies should investigate whether the way in which principals use observation protocols results in better, more useful observation scores. Tables are appended.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: California