NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED511781
Record Type: Non-Journal
Publication Date: 2009-Oct
Pages: 131
Abstractor: As Provided
Reference Count: 82
What to Do when Data Are Missing in Group Randomized Controlled Trials. NCEE 2009-0049
Puma, Michael J.; Olsen, Robert B.; Bell, Stephen H.; Price, Cristofer
National Center for Education Evaluation and Regional Assistance
This NCEE Technical Methods report examines how to address the problem of missing data in the analysis of data in Randomized Controlled Trials (RCTs) of educational interventions, with a particular focus on the common educational situation in which groups of students such as entire classrooms or schools are randomized. Missing outcome data are a problem for two reasons: (1) the loss of sample members can reduce the power to detect statistically significant differences, and (2) the introduction of non-random differences between the treatment and control groups can lead to bias in the estimate of the intervention's effect. The report reviews a selection of methods available for addressing missing data, and then examines their relative performance using extensive simulations that varied a typical educational RCT on three dimensions: (1) the amount of missing data; (2) the level at which data are missing--at the level of whole schools (the assumed unit of randomization) or for students within schools; and, (3) the underlying missing data mechanism. The performance of the different methods is assessed in terms of bias in both the estimated impact and the associated standard error. Appendices include: (1) Missing Data Bias as a Form of Omitted Variable Bias; (2) Resources for Using Multiple Imputation; (3) Specifications for Missing Data Simulations; (4) Full Set of Simulation Results; and (5) Standards for Judging the Magnitude of the Bias for Different Missing Data Methods. (Contains 12 tables, 4 exhibits and 88 footnotes.)
National Center for Education Evaluation and Regional Assistance. Available from: ED Pubs. P.O. Box 1398, Jessup, MD 20794-1398. Tel: 877-433-7827; Web site:
Publication Type: Reports - Evaluative
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Education Evaluation and Regional Assistance (ED)
IES Funded: Yes
IES Cited: ED544189; ED560820; ED544269