ERIC Number: EJ1262097
Record Type: Journal
Publication Date: 2020
Pages: 30
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1934-5747
EISSN: N/A
Using Data from Randomized Trials to Assess the Likely Generalizability of Educational Treatment-Effect Estimates from Regression Discontinuity Designs
Bloom, Howard; Bell, Andrew; Reiman, Kayla
Journal of Research on Educational Effectiveness, v13 n3 p488-517 2020
This article assesses the likely generalizability of educational treatment-effect estimates from regression discontinuity designs (RDDs) when treatment assignment is based on academic pretest scores. Our assessment uses data on outcome and pretest measures from six educational experiments, ranging from preschool through high school, to estimate RDD generalization bias. We then compare those estimates (reported as standardized effect sizes) with the What Works Clearinghouse (WWC) standard for acceptable bias size ([less than or equal to] 0.05[delta]) for two target populations, one spanning a half-standard deviation pretest-score range and another spanning a full-standard deviation pretest-score range. Our results meet this standard for all 18 study/outcome/pretest scenarios examined given the narrower target population, and for 15 scenarios given the broader target population. Fortunately, two of the three exceptions represent pronounced "ceiling effects" that can be identified empirically, making it possible to avoid unwarranted RDD generalizations, and the third exception is very close to the WWC standard. [For the corresponding grantee submission, see ED604658.]
Descriptors: Data Use, Randomized Controlled Trials, Research Design, Regression (Statistics), Outcomes of Education, Pretests Posttests, Effect Size, Statistical Bias, Preschool Children, Elementary School Students, High School Students, Generalization
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Location: North Carolina; New York (New York)
IES Funded: Yes
Grant or Contract Numbers: R305D140012