ERIC Number: EJ1015751
Record Type: Journal
Publication Date: 2013-Nov
Pages: 12
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1531-7714
EISSN: N/A
Comparing Propensity Score Methods in Balancing Covariates and Recovering Impact in Small Sample Educational Program Evaluations
Stone, Clement A.; Tang, Yun
Practical Assessment, Research & Evaluation, v18 n13 Nov 2013
Propensity score applications are often used to evaluate educational program impact. However, various options are available to estimate both propensity scores and construct comparison groups. This study used a student achievement dataset with commonly available covariates to compare different propensity scoring estimation methods (logistic regression, boosted regression, and Bayesian logistic regression) in combination with different methods for constructing comparison groups (nearest-neighbor matching, optimal matching, weighting) relative to balancing pre-existing differences and recovering a simulated treatment effect in small samples. Results indicated that applied researchers evaluating program impact should first consider use of standard logistic regression methods with nearest-neighbor or optimal matching or boosted regression in combination with propensity score weighting. Advantages and disadvantages of the methods are discussed. (Contains 3 tables.)
Descriptors: Comparative Analysis, Probability, Sample Size, Program Evaluation, Regression (Statistics), Bayesian Statistics, Statistical Analysis, Middle School Students, Experimental Groups, Control Groups
Center for Educational Assessment. 813 North Pleasant Street, Amherst, MA 01002. e-mail: pare@umass.edu; Tel: 413-577-2180; Web site: https://scholarworks.umass.edu/pare
Publication Type: Journal Articles; Reports - Research
Education Level: Middle Schools; Junior High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A