ERIC Number: ED489855
Record Type: Non-Journal
Publication Date: 2005-Jun-22
Statistical Power for Random Assignment Evaluations of Education Programs
Schochet, Peter Z.
Mathematica Policy Research, Inc.
This paper examines issues related to the statistical power of impact estimates for experimental evaluations of education programs. The focus is on "group-based" experimental designs, because many studies of education programs involve random assignment at the group level (for example, at the school or classroom level) rather than at the student level. The clustering of students within groups (units) generates design effects that considerably reduce the precision of the impact estimates, because the outcomes of students within the same schools or classrooms tend to be correlated (that is, are not independent of each other). This, statistical power is a concern for these evaluations. The report is organized into five sections. First, it discusses general issues for a statistical power analysis, including procedures for assessing appropriate precision levels. Second, it discusses reasons that a clustered design reduces the statistical power of impact estimates and provides a simple mathematical formulation of the problem. Third, it presents procedures that can be used to reduce design effects. Fourth, it provides power calculations for impact estimates under various design options and parameter assumptions, and finally, it presents conclusions. Appended is: Values for Factor (.) in Equation (2). (Contains 8 tables.) [This report was submitted by Mathematica Policy Research, Inc. to the Institute of Education Sciences.]
Descriptors: Statistical Analysis, Evaluation Methods, Program Evaluation, Research Design, Research Methodology, Educational Research, Elementary Schools, Early Childhood Education, Correlation, Sampling, Sample Size, Effect Size, Computation
Mathematica Policy Research, Inc., P.O. Box 2393, Princeton, NJ 08543-2393. Tel: 609-799-3535; Fax: 609-799-0005; e-mail: email@example.com; Web site: http://www.mathematica-mpr.com/publications/.
Publication Type: Reports - Research
Education Level: Elementary Education
Authoring Institution: Mathematica Policy Research, Princeton, NJ.; Institute of Education Sciences (ED), Washington, DC.
IES Cited: ED511782; ED503448; ED511789