ERIC Number: ED533141
Record Type: Non-Journal
Publication Date: 2012-Jun
Abstractor: As Provided
Reference Count: 14
Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials. Working Paper
Deke, John; Dragoset, Lisa
Mathematica Policy Research, Inc.
The regression discontinuity design (RDD) has the potential to yield findings with causal validity approaching that of the randomized controlled trial (RCT). However, Schochet (2008a) estimated that, on average, an RDD study of an education intervention would need to include three to four times as many schools or students as an RCT to produce impacts with the same level of statistical precision. We extend the work of Schochet by empirically assessing the effect on sample size requirements of accounting for selection of an optimal bandwidth and the adjustment for random misspecification error, both of which are needed to estimate consistent RDD impacts and control the Type I error rate. We used data from four previously published education studies covering more than 25,000 students in kindergarten to grade 9 in 24 states and 330 schools to calculate empirical estimates of the RDD design effect taking into account these additional factors. We find that an RDD study needs between 9 and 17 times as many schools or students as an RCT to produce an impact with the same level of statistical precision, and that the need for a large sample is driven primarily by bandwidth selection, not adjusting for random misspecification error. (Contains 9 tables and 5 footnotes.)
Descriptors: Research Design, Elementary Secondary Education, Regression (Statistics), Educational Research, Sampling, Error Patterns, Computation, Program Evaluation, Intervention, Pretests Posttests, Effect Size
Mathematica Policy Research, Inc. P.O. Box 2393, Princeton, NJ 08543-2393. Tel: 609-799-3535; Fax: 609-799-0005; e-mail: firstname.lastname@example.org; Web site: http://www.mathematica-mpr.com
Publication Type: Reports - Research
Education Level: Elementary Secondary Education; Grade 1; Grade 2; Grade 3; Grade 4; Grade 5; Grade 6; Grade 7; Grade 8; Grade 9; Kindergarten
Authoring Institution: Mathematica Policy Research, Inc.