ERIC Number: ED560247
Record Type: Non-Journal
Publication Date: 2015-Apr
Abstractor: As Provided
Reference Count: 41
Learning "about" and "from" Variation in Program Impacts Using Multisite Trials. MDRC Working Papers on Research Methodology
Raudenbush, Stephen W.; Bloom, Howard S.
The present paper, which is intended for a diverse audience of evaluation researchers, applied social scientists, and research funders, provides a broad overview of the conceptual and statistical issues involved in using multisite randomized trials to learn "about" and "from" variation in program effects across "individuals," across policy-relevant and theoretically relevant "subgroups" of individuals, and across program sites. Learning about variation in program effects involves detecting and quantifying this variation. Learning from variation in program effects involves studying the factors which predict or explain it. The paper is divided into four sections, plus a brief final discussion. The first section introduces the concepts and issues involved. Section 2 focuses on detecting and quantifying variation in effects of program "assignment," which are often referred to as effects of intent to treat (ITT). Section 3 extends the discussion to variation in effects of program "participation," which are often referred to as a complier average causal effect (CACE) or a local average treatment effect (LATE). Section 4 considers "moderators" of program effects (individual-level or site-level factors that influence the sign and magnitude of these effects) and "mediators" of program effects (individual-level or site-level "mechanisms" by which these effects are produced). Appendices include: (1) Characterizing the Impact of an Intervention on the Mean and Variance of Outcomes and Program Impacts for a Population of Individuals; (2) Derivation of the HLM [hierarchical linear model] Estimator of the Cross-Site Mean and Variance of Program Impacts.
Descriptors: Program Effectiveness, Research Methodology, Statistical Analysis, Differences, Intention, Participation, Influences, Identification, Computation, Hierarchical Linear Modeling
MDRC. 16 East 34th Street 19th Floor, New York, NY 10016-4326. Tel: 212-532-3200; Fax: 212-684-0832; e-mail: email@example.com; Web site: http://www.mdrc.org
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Spencer Foundation; William T. Grant Foundation
Authoring Institution: MDRC