ERIC Number: ED161897
Record Type: RIE
Publication Date: 1978-Mar
Reference Count: 0
Coder Drift: A Reliability Problem for Teacher Observations.
Marston, Paul T.; And Others
The results of two experiments support the hypothesis of "coder drift" which is defined as change that takes place while trained coders are using a system for a number of classroom observation sessions. The coding system used was a modification of the low-inference Flanders System of Interaction Analysis which calls for assigning observed teacher or student behaviors to ten relatively objective categories. The situations rated in these experiments consisted of videotapes of social studies classes which were conducted by experienced teachers. In each experiment, pairs of coders viewed the same series of ten videotapes, either working with their partners or working in separate rooms. In one condition, the pairs worked together for the first five sessions and apart for the other five. In the other condition, that order was reversed. Pairs working together for the first five sessions showed higher reliabilities than they did when they worked apart, and pairs working apart for the first five sessions showed even lower reliability. The same effect was found for five different methods of computing reliability. Randomly matched individuals showed no drift effects, nor did test-retest reliabilities between pairs working together and those working apart. (Author/CTM)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC.
Authoring Institution: Texas Univ., Austin. Research and Development Center for Teacher Education.
Identifiers: Coder Drift; Flanders System of Interaction Analysis; Interrater Reliability
Note: Paper presented at the Annual Meeting of the American Educational Research Association (62nd, Toronto, Ontario, Canada, March 27-31, 1978); For related document, see TM 007479