Download full text
Download full text
ERIC Number: ED562850
Record Type: Non-Journal
Publication Date: 2014
Reference Count: 5
What Is the Minimum Information Needed to Estimate Average Treatment Effects in Education RCTs?
Schochet, Peter Z.
Society for Research on Educational Effectiveness
Randomized controlled trials (RCTs) are considered the "gold standard" for evaluating an intervention's effectiveness. Recently, the federal government has placed increased emphasis on the use of opportunistic experiments. A key criterion for conducting opportunistic experiments, however, is that there is relatively easy access to data about key outcomes. This paper will address the following research question: What is the "minimum" amount of information that researchers can request from state longitudinal data systems (SLDS) staff to obtain unbiased estimates of average treatment effects (ATEs) and their standard errors for the full population and key population subgroups? The paper will consider non-clustered designs where (1) students are randomized to a treatment and control group, (2) clustered designs where units (such as schools or classrooms) are randomized, and (3) both stratified designs where random assignment is conducted within blocks (such as school districts or schools) and non-stratified designs. The paper will focus on the estimation of ATEs on a continuous student achievement test score that is available in the SLDS data. Also, the author will consider impact estimation models that adjust for baseline covariates and those that do not. The study results report that RCTs will be feasible only if administrative records data are readily available to conduct such analyses. Yet, obtaining these data can be difficult and time-consuming due to data access issues and staff availability.
Descriptors: Randomized Controlled Trials, Outcomes of Treatment, Intervention, Program Effectiveness, Longitudinal Studies, Data, Access to Information, Error of Measurement, Control Groups, Experimental Groups, Academic Achievement, Scores, Statistical Analysis
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: firstname.lastname@example.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: N/A