NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED567017
Record Type: Non-Journal
Publication Date: 2016
Pages: 11
Abstractor: ERIC
Reference Count: 11
Combining Propensity Score Methods and Complex Survey Data to Estimate Population Treatment Effects
Stuart, Elizabeth A.; Dong, Nianbo; Lenis, David
Society for Research on Educational Effectiveness
Complex surveys are often used to estimate causal effects regarding the effects of interventions or exposures of interest. Propensity scores (Rosenbaum & Rubin, 1983) have emerged as one popular and effective tool for causal inference in non-experimental studies, as they can help ensure that groups being compared are similar with respect to a large set of observed characteristics. However, little work has investigated how best to combine propensity scores and complex survey data to estimate population treatment effects. This has led to confusion in the literature, with many applied researchers using inappropriate methods or claiming representativeness of study results when the analysis does not warrant such claims (DuGoff, Schuler and Stuart, 2014; Ridgeway, Kovalchik, Griffin, and Kabeto, 2015). One way to think about the complication of estimating population treatment effects using data from a complex survey is that when there are heterogeneities in treatment assignment, sample selection probabilities, and treatment effects, failure to take into account sampling weights might cause biased population treatment effect estimates. Ignoring sampling weights leads mainly to external validity bias, which occurs when people inappropriately make inferences from the unrepresentative analytic sample to the target population. This work aims to clarify the results and recommendations regarding the use of propensity scores with complex survey data. Tables are appended.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: Kindergarten; Primary Education; Early Childhood Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Assessments and Surveys: Early Childhood Longitudinal Survey