ERIC Number: ED068534
Record Type: RIE
Publication Date: 1971-Sep
Reference Count: 0
Using Generalizability Theory and Multifacet Designs in the Validation of a Classroom Observation Instrument.
Baker, J. Philip
The usefulness of generalizability theory in assessing the reliability of classroom observation instruments is illustrated, with a new index of reliability, called the coefficient of generalizability, given as an index of how well one can generalize from the instrument to the universe score according to the conditions of observation. Data from an instrument assessing verbal behavior are reanalyzed using generalizability theory. Three raters observed two classes of five student teachers, and three facets (class, occasion, and rater) were specified for study. Analysis of variance yielded the components of variance due to these facets, and ratio of the estimated universe. Score variance to the observed-score variance was computed as the coefficient of generalizability. The possible effect of various hypothetical conditions of observation on the coefficient of generalizability was considered. The results showed the coefficient of generalizability for one rater observing one class on one day was near zero for most of the summary variables for the instrument, but it increased when the number of observers and days of observation were increased. This was directly attributable to rater bias. Different observer training procedures might improve the usefulness of the instrument. (Author/LH)
Publication Type: N/A
Education Level: N/A
Sponsor: Office of Education (DHEW), Washington, DC.
Authoring Institution: Stanford Univ., CA. Stanford Center for Research and Development in Teaching.