ERIC Number: ED492893
Record Type: Non-Journal
Publication Date: 2006-Jul
Reference Count: 4
Using Classroom Artifacts to Measure Instructional Practice in Middle School Science: A Two-State Field Test. CSE Technical Report 690
Borko, Hilda; Stecher, Brian M.
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
This report presents findings from two investigations of the use of classroom artifacts to measure the presence of reform-oriented teaching practices in middle-school science classes. It complements previous research on the use of artifacts to describe reform-oriented teaching practices in mathematics. In both studies, ratings based on collections of artifacts assembled by teachers following directions in the "Scoop Notebook" are compared to judgments based on other sources of information, including direct classroom observations and transcripts of discourse recorded during classroom observations. For this purpose, we developed descriptions of 11 dimensions of reform-oriented science instruction, and procedures for rating each on a dimension-specific five-point scale. Two investigations were conducted. In 2004, data were collected from 39 middle-school science teachers in two states. Each teacher completed a Scoop Notebook, each was observed by a single observer on two or three occasions, and eight of the teachers were also audio-taped, allowing us to create transcripts of classroom discourse. In 2005, 21 middle-school mathematics teachers participated in a similar study, in which each teacher was observed by a pair of observers, but no audio-taping occurred. All data sources were rated independently on each of 11 dimensions. In addition, independent ratings were made using combinations of data sources. The person who observed in a classroom also reviewed the Scoop Notebook and assigned a "gold standard" rating reflecting all the information available from the Notebook and the classroom observations. Combined ratings were also assigned based on the transcripts and notebooks, and based on the observations and transcripts. The results of these field studies suggest that the Scoop Notebook is a reasonable tool for describing instructional practice in broad terms. For example, it could be useful for providing an indication of changes in instruction over time that occur as a result of program reform efforts. There was a moderate degree of correspondence between judgments of classroom practice based on the Scoop Notebook and judgments based on direct classroom observation. Correspondence was particularly high for dimensions that did not exhibit great variation from one day to the next. Furthermore, judgments based on the Scoop Notebook corresponded moderately well to our "gold standard" ratings, which included all the information we had about practice. The following are appended: (1) Teacher Reflection Questions; and (2) Scoring Guide. (Contains 28 tables and 14 footnotes.)
Descriptors: Teaching Methods, Science Instruction, Investigations, Mathematics, Instructional Materials, Middle School Teachers, Data Collection, Observation, Reliability, Validity, Scoring Rubrics, Homework
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org.
Publication Type: Reports - Evaluative; Tests/Questionnaires
Education Level: Middle Schools
Sponsor: Institute of Education Sciences (ED), Washington, DC.
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.
Identifiers: Following Directions; California; Colorado