ERIC Number: ED548247
Record Type: Non-Journal
Publication Date: 2014-Nov
Abstractor: As Provided
Reference Count: 31
Estimating Causal Effects of Education Interventions Using a Two-Rating Regression Discontinuity Design: Lessons from a Simulation Study
Porter, Kristin E.; Reardon, Sean F.; Unlu, Fatih; Bloom, Howard S.; Robinson-Cimpian, Joseph P.
A valuable extension of the single-rating regression discontinuity design (RDD) is a multiple-rating RDD (MRRDD). To date, four main methods have been used to estimate average treatment effects at the multiple treatment frontiers of an MRRDD: the "surface" method, the "frontier" method, the "binding-score" method, and the "fuzzy instrumental variables" method. This paper uses a series of simulations to evaluate the relative performance of each of these four methods under a variety of different data-generating models. Focusing on a two-rating RDD (2RRDD), the methods are compared in terms of their bias, precision, and mean squared error when implemented as they most likely would be in practice--using optimal bandwidth selection. The lessons learned from the simulations to a real-world example that uses data from a study of an English learner reclassification policy are also applied. Overall, this paper makes valuable contributions to the literature on MRRDDs in that it makes concrete recommendations for choosing among MRRDD estimation methods, for implementing any chosen method using local linear regression, and for providing accurate statistical inferences. Appended are: (1) Computing Average Frontier Effects for Models 3 and 4 in Our Simulations; (2) Estimation in Theory: Using Full Information; (3) Deriving Correct Model Specifications for the Fuzzy IV Method; (4) Domain or Bandwidth Selection; and (5) Simulation Results When Using Local Linear Regression, Not Including Other Rating as a Covariate.
Descriptors: Regression (Statistics), Research Design, Quasiexperimental Design, Research Methodology, Causal Models, Influences, Intervention, Computation, Simulation, Statistical Bias, Accuracy, Error of Measurement, Statistical Inference
MDRC. 16 East 34th Street 19th Floor, New York, NY 10016-4326. Tel: 212-532-3200; Fax: 212-684-0832; e-mail: firstname.lastname@example.org; Web site: http://www.mdrc.org
Publication Type: Numerical/Quantitative Data; Reports - Research
Education Level: N/A
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: MDRC
IES Funded: Yes
Grant or Contract Numbers: R305D100027