ERIC Number: ED574544
Record Type: Non-Journal
Publication Date: 2015
Abstractor: As Provided
Reference Count: 61
Assessing Tradeoffs between Observational and Experimental Designs for Charter School Research. Program on Education Policy and Governance Working Papers Series. PEPG 15-04
Ackerman, Matthew; Egalite, Anna J.
Program on Education Policy and Governance
When lotteries are infeasible, researchers must rely on observational methods to estimate charter effectiveness at raising student test scores. Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the sensitivity of CREDO's findings to its methodology has not been subject to a thorough, independent evaluation. Our analysis of the CREDO estimates of charter effectiveness in Florida between 2001 and 2009 reveals four main findings that increase confidence in its results: (1) Its use of multiple observations when matching students does not materially affect results; (2) Student participation in programs such as special education and free/reduced price lunch is inconsistently measured across sectors, but matching on these variables only modestly affects CREDO's estimates; (3) Exogenous instrumental variable estimates of charter effectiveness produce qualitatively similar results to CREDO's observational estimates; and (4) Impact estimates differ for oversubscribed and undersubscribed charters, which helps explain why lottery-based studies tend to find larger charter impacts than CREDO does. The following are appended: (1) Other Potential Concerns with CREDO's Methodology; (2) Differential Program Participation; (3) Analysis of Transfer Students Only; and (4) Summary Statistics for Over and Undersubscribed Charter Schools.
Descriptors: Charter Schools, Observation, Special Education, Lunch Programs, School Effectiveness, Transfer Students, Educational Research, Achievement Tests, Scores, Educational Policy, Achievement Gains, Institutional Evaluation, Evaluation Methods, Statistical Analysis, Evaluation Problems, Error of Measurement, Control Groups, Experimental Groups, Comparative Analysis, Classification, Limited English Speaking, Elementary Secondary Education, Standardized Tests, State Standards
Program on Education Policy and Governance. Harvard University, Kennedy School of Government, 79 John F. Kennedy Street, Taubman 304, Cambridge, MA 02138. Tel: 617-495-7976; Fax: 617-496-4428; e-mail: firstname.lastname@example.org; Web site: http://www.hks.harvard.edu/pepg
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Authoring Institution: Harvard University, Program on Education Policy and Governance
Identifiers - Location: Florida; California (Stanford)
Identifiers - Assessments and Surveys: Florida Comprehensive Assessment Test