ERIC Number: ED148893
Record Type: RIE
Publication Date: 1977-Feb
Reference Count: 0
The Study of Reliability in Observational Data. Occasional Paper 19.
Stallings, Jane A.; Giesen, Philip A.
Observer reliability and the confusability of codes, two sources of error in the collection of classroom observational data, are examined. Confusability is defined as the extent to which one code is mistakenly recorded as another code. Observational data were collected in each of 172 first grade and 171 third grade Follow Through and comparison classrooms in urban and rural locations throughout the U.S. One section of the Stanford Research Institute's classroom observation instrument was analyzed for confusability of codes. Twenty simulated classroom situations were videotaped and coded by 63 trained observers. Matrices listing all the codes were constructed for each observer. The observer's coding was recorded so that their errors would be detectable, and coding errors were analyzed. Separate tables presented each observer's criterion video tape accuracy rate by sponsor, site, and grade level. Analyses of these matrices were also used to study the confusability of the codes. The results showed that the high rate of confusability of several observers could have been caused by overlapping code definitions, poor video tape examples, or inadequate training. (MV)
Descriptors: Bias, Classroom Observation Techniques, Classroom Research, Codification, Data Collection, Error Patterns, Grade 1, Grade 3, Observation, Primary Education, Recordkeeping, Reliability, Research Problems, Scoring, Simulation, Videotape Recordings
Phi Delta Kappa, Eighth and Union, Box 789, Bloomington, Indiana 47401 ($1.50)
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: Phi Delta Kappa, Bloomington, IN. Center on Evaluation and Research.
Note: Revised version of a paper presented at the Annual Meeting of the American Educational Research Association (58th, Chicago, Illinois, April 15-19, 1974)