ERIC Number: EJ1146235
Record Type: Journal
Publication Date: 2013
Abstractor: As Provided
Using Generalizability Theory as a Framework for Informing Measurement Issues in Middle School Settings
Ing, Marsha; Shih, Jeffrey C.
Middle Grades Research Journal, v8 n2 p25-39 2013
There are situations within middle school settings where measurements of students and teachers are used for high-stakes decisions. For example, student performance is used as an indicator of teacher quality or determines student eligibility for particular types of support services. Given the high-stakes nature of these types of assessments, understanding the strengths and limitations of the measurement is essential. Generalizability theory is a method to help better understand reliability and can be used to inform and improve measurement issues in middle school settings. The purpose of this article is to create a more informed consumer of measurement issues in middle school settings by focusing on the reliability of a particular measurement scenario: observation of middle school teacher behaviors. Teacher observations might be conducted by other teachers, principals, or other administrators or researchers as one indicator of instructional quality. We describe how a 1-facet crossed design and 2-facet crossed design can be applied to different observational protocols and offer considerations for those interested in conducting a generalizability study. We emphasize that no single design will apply to all generalizability studies and findings for a particular observational measure will not necessarily generalize to other observational measures. Instead, the researcher must consider the purpose and use for their particular observational measure.
Descriptors: Generalizability Theory, Middle School Teachers, Teacher Behavior, Research Design, Error of Measurement, Teacher Evaluation, Teacher Effectiveness, Classroom Observation Techniques, Reliability
IAP - Information Age Publishing, Inc. P.O. Box 79049, Charlotte, NC 28271-7047. Tel: 704-752-9125; Fax: 704-752-9113; e-mail: firstname.lastname@example.org; Web site: http://www.infoagepub.com/middle-grades-research-journal.html
Publication Type: Journal Articles; Reports - Descriptive
Education Level: Middle Schools; Secondary Education; Junior High Schools
Authoring Institution: N/A
Grant or Contract Numbers: N/A