ERIC Number: ED517990
Record Type: Non-Journal
Publication Date: 2010
Reference Count: 22
The Implications of Teacher Selection and Teacher Effects in Individually Randomized Group Treatment Trials
Weiss, Michael J.
Society for Research on Educational Effectiveness
Randomized experiments have become an increasingly popular design to evaluate the effectiveness of interventions in education (Spybrook, 2008). Many of the interventions evaluated in education are delivered to groups of students, rather than to individuals. Experiments designed to evaluate programs delivered at the group level often randomize intact groups, such as classes or schools, to treatment and control conditions in order to obtain unbiased estimates of intervention effects (H. S. Bloom, 2006; H. S. Bloom, et al., 2008). A growing body of research has discussed the analytics of such group-randomized trials (GRTs), or cluster-randomized trials, as well as the appropriate interpretation of impact estimates (H. S. Bloom, 2006; H. S. Bloom, Richburg-Hayes, & Black, 2007; H. S. Bloom, et al., 2008; Hedges, 2007; Hedges & Hedberg, 2007; Konstantopoulos, 2008a, 2008b; Spybrook, 2008). This work describes how, in studies of classroom-level interventions that do not randomize teachers to experimental conditions, it will be unclear whether measured differences between program and control group students are a result of the core components of the intervention or the teachers (i.e., teacher effects). This potential confounding is a major concern if teachers are sorted into experimental conditions in such a way that they differ on variable(s) that are related to their effectiveness. This work attempts to make clear the correct interpretation of typically calculated "program impacts" in this situation. In addition, using the magnitude of estimated teacher effects from prior research, this work demonstrates that if teachers are not randomly assigned to experimental conditions, then it is significantly more difficult to establish whether the intervention "works" or if the types of teachers selected to teach in intervention classrooms are simply more/less effective than their control group counterparts. The implications may be quite serious in terms of the usefulness of such studies' findings. (Contains 1 table, 2 figures and 9 footnotes.
Descriptors: Intervention, Teacher Effectiveness, Control Groups, Experimental Groups, Educational Research, Research Design, Community Colleges, Two Year College Students, College Faculty, Program Effectiveness
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: firstname.lastname@example.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: Postsecondary Education; Two Year Colleges
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: New York