NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1023855
Record Type: Journal
Publication Date: 2014-Jun
Pages: 15
Abstractor: As Provided
ISSN: ISSN-1534-5084
Measuring Rater Reliability on a Special Education Observation Tool
Semmelroth, Carrie Lisa; Johnson, Evelyn
Assessment for Effective Intervention, v39 n3 p131-145 Jun 2014
This study used generalizability theory to measure reliability on the Recognizing Effective Special Education Teachers (RESET) observation tool designed to evaluate special education teacher effectiveness. At the time of this study, the RESET tool included three evidence-based instructional practices (direct, explicit instruction; whole-group instruction; and discrete trial teaching) as the basis for special education teacher evaluation. Five raters participated in two sessions to evaluate special education classroom instruction collected from two school years, via the Teachscape 360-degree video system. Data collected from raters were analyzed in a two-facet "partially" nested design where occasions (o) were nested within teachers (t), o:t, and crossed with raters (r), {o:t} x r. Results from this study are in alignment with similar studies that found multiple observations and multiple raters are critical for ensuring acceptable levels of measurement score reliability. Considerations for the feasibility of practice should be observed in future reliability and validity studies on the RESET tool, and further work is needed to address the lack of research on rater reliability issues within special education teacher evaluation.
SAGE Publications and Hammill Institute on Disabilities. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Reports - Research; Journal Articles
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Idaho
Grant or Contract Numbers: N/A