ERIC Number: ED453268
Record Type: RIE
Publication Date: 2001-Apr
Examining Differences in Teachers' and Researchers' Understanding of an Instructional Log.
Camburn, Eric; Correnti, Richard; Taylor, James
This study, part of the "Study of Instructional Improvement," assessed the validity of quantitative measures of reading comprehension instruction derived from a language arts instructional log. The log was designed to measure the language arts instructional experiences individual students received over time, and measures from the log were to be used as independent variables in statistical models of student achievement growth. Data are from two nested studies. In the first study, 91 teachers completed language arts instructional logs every school day for a 3-month period. In a subsidiary study, a subset of 31 of these teachers were observed by 2 researchers on 1 day of instruction during the 3 months. This study yielded 3 independent ratings (teacher and 2 observers) of 23 reading comprehension lessons. Measures of reading comprehension instruction were constructed from the first study's data set using factor analysis, and then proxy measures of reading comprehension instruction were created from the observations from the second study. Initial assessments of the validity of the reading comprehension measure were made using two indicators that gauged inter-rater agreement in the second study. The validity of the measures was further studied by evaluating qualitative evidence form the observations. The factor analyses of the data from the first study provide fairly traditional quantitative evidence of the construct validity of the measures. The measures of inter-rater agreement go beyond to give a general indication of the degree to which there was shared understanding of what occurred during the classroom observations. The three analyses yielded markedly different pictures of the validity of the reading comprehension instruction measures, including the fact that teachers generally reported much higher levels of reading comprehension activity than observers, and that raters marked different items about one quarter of the time. Appendixes contain the Language Arts Log and a glossary for the log. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: Department of Education, Washington, DC.; National Science Foundation, Arlington, VA.
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Seattle, WA, April 10-14, 2001).