NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1204987
Record Type: Journal
Publication Date: 2019-Mar
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1098-2140
EISSN: N/A
Variance Estimation in Evaluations with No-Shows: A Comparison of Methods
Litwok, Daniel; Peck, Laura R.
American Journal of Evaluation, v40 n1 p104-118 Mar 2019
In experimental evaluations of policy interventions, the so-called Bloom adjustment is commonly used to estimate the impact of the treatment on the treated. It does so by rescaling the estimated impact of the intention to treat--that is, the overall treatment-control group difference in outcomes for the entire experimental sample--by the percentage of cases that took up the treatment offer. The practice of also rescaling the variance, as is common in the literature, imposes simplifying assumptions that may lead to biased variance estimates. We compare variances using the Bloom adjustment to variances that capture all the estimation error. While the difference between these variances is negligible in three experimental evaluations, we highlight three conditions that could result in larger relative bias: large impacts, large variability in compliance, and/or substantial endogeneity bias. The presence of these conditions could potentially result in a different conclusion for policy or practice. [An earlier version of this article was prepared, in collaboration with Luke Keele, with federal funding from the U.S. Department of Labor, Employment and Training Administration, under Contract Number GS10F0086J/DOLF091A21626.]
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A