ERIC Number: ED252545
Record Type: RIE
Publication Date: 1980-Feb
Reference Count: 0
Interobserver Agreement for the Observation Procedures for the DMP and WDRSD Observers. Descriptive Study. Phase IV. Project Paper 79-25. Parts 1 and 2.
Webb, Norman L.
This project paper reports the interobserver agreements and reliabilities for the observation procedures used in the Descriptive Study of Phase IV of the Individually Guided Education Evaluation Project. Only data from four observers--at the two Developing Mathematical Processes Schools and the two Wisconsin Design for Reading Skills Development Schools, grades 2 and 5--were used. Interobserver agreements and reliabilities were computed for each of three situations: training session, retraining session, and on-site visits. For each pair of observers and for each category, marginal agreements were computed. For the category sets--General Content, Pacing, Grouping, Materials, Learner Moves, and Interactions--Category Marginal Agreements and Cohen's Kappa were computed. Generalizability coefficients for estimates of reliability were computed for the training and retraining sessions for groups of three observers observing two or three classes for two days. The results indicated that there was high interobserver agreement at the three different check times on the categories of self-paced, other-paced, individual, large group, paper/pencil, engaged-written, engaged-covert, and non-engaged off task. The low interobserver agreements were explained mainly as a result of a low frequency of occurrence. The generalizability coefficients for total engagement and total non-engagement were acceptable. (Author/BS)
Descriptors: Classroom Observation Techniques, Elementary Education, Generalizability Theory, Grade 2, Grade 5, Individualized Instruction, Interrater Reliability, Mathematics Instruction, Programed Instruction, Reading Instruction, Student Behavior, Student Evaluation, Time on Task, Training Methods
Publication Type: Reports - Research; Numerical/Quantitative Data
Education Level: N/A
Sponsor: National Inst. of Education (ED), Washington, DC.
Authoring Institution: Wisconsin Univ., Madison. Research and Development Center for Individualized Schooling.
Identifiers: Developing Mathematical Processes; Individually Guided Education; Kappa Coefficient; Wisconsin Design for Reading Skill Development
Note: Report from the Individually Guided Education (IGE) Project. For related documents, see TM 840 805-812 and TM 850 019.