NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ862322
Record Type: Journal
Publication Date: 2009
Pages: 29
Abstractor: As Provided
Reference Count: 37
ISSN: ISSN-0193-841X
An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations
Schochet, Peter Z.
Evaluation Review, v33 n6 p539-567 2009
In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying confirmatory and exploratory analyses in study protocols, delineating confirmatory outcome domains, conducting t tests on composite domain outcomes, and applying multiplicity corrections to composites across domains to obtain summative impact evidence. The article presents statistical background and discusses multiplicity issues for subgroup analyses, designs with multiple treatments, and reporting. (Contains 4 tables and 5 notes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A