NotesFAQContact Us
Search Tips
ERIC Number: ED514995
Record Type: Non-Journal
Publication Date: 2009
Pages: 8
Abstractor: ERIC
Reference Count: 5
On the Importance of Reliable Covariate Measurement in Selection Bias Adjustments Using Propensity Scores
Steiner, Peter M.; Cook, Thomas D.; Shadish, William R.
Society for Research on Educational Effectiveness
This paper investigates how bias reduction was affected when different degrees of measurement error were systematically introduced into the measures constituting the final estimated propensity score (PS), the PS only for the set of effective covariates and the PS only for the ineffective ones. Since there was already some error in the Shadish et al. covariate measures, a more complex simulation was also done without this source of error. In many ways, this last analysis is the most important. This study uses data from a within-study comparison as the basis for a simulation study. The results suggest that the covariates deemed to be most effective in reducing selection bias should be reliably measured. Failure to do so will reduce the bias reduction achieved. In contrast, the reliability of ineffective covariates has minimal effect on bias reduction. Further, the larger the set of interrelated covariates used to control for selection bias the less sensitive is bias reduction to measurement error. But it is crucial to include the singly most effective covariates within this covariate set. The authors did not find any method specific differences in attenuation rates. When the same covariates are used in all analyses, PS methods show basically the same sensitivity to measurement errors in covariates as ANCOVA (analysis of covariance). (Contains 2 figures.)
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)