ERIC Number: ED471814
Record Type: Non-Journal
Publication Date: 2002-Jun
Reference Count: N/A
Can Nonexperimental Comparison Group Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs? MDRC Working Papers on Research Methodology.
Bloom, Howard S.; Michalopoulos, Charles; Hill, Carolyn J.; Lei, Ying
A study explored which nonexperimental comparison group methods provide the most accurate estimates of the impacts of mandatory welfare-to-work programs and whether the best methods work well enough to substitute for random assignment experiments. Findings were compared for nonexperimental comparison groups and statistical adjustment procedures with those for experimental control groups from a large-sample, six-state random assignment experiment--the National Evaluation of Welfare-to-Work Strategies. The methods were assessed in terms of their ability to estimate program impacts on annual earnings during short-run and medium-run follow-up periods. Findings with respect to the first issue suggested in-state comparison groups perform somewhat better than out-of-state or multi-state, especially for medium- run impact estimates; a simple difference of means or ordinary least squares regression can perform as well or better than more complex methods when used with a local comparison group; impact estimates for out-of-state or multi-state comparison groups are not improved substantially by more complex estimation procedures but are improved somewhat when propensity score methods are used to eliminate comparison groups that are not balanced on their baseline characteristics. Findings with respect to the second issue indicated the best methods did not work well enough to replace random assignment. Statistical analyses are appended. (Contains 51 references.) (YLB)
Descriptors: Adult Education, Comparative Analysis, Control Groups, Error of Measurement, Estimation (Mathematics), Experimental Groups, Groups, Least Squares Statistics, Program Evaluation, Regression (Statistics), Research Design, Research Methodology, Sampling, Statistical Analysis, Statistical Studies, Welfare Recipients, Welfare Reform, Welfare Services
Manpower Demonstration Research Corporation, 16 East 34 Street, New York, New York 10016. Tel: 212-532-3200; Web site: http://www.mdrc.org. For full text: http://www.mdrc.org/Reports2002/nonexperimental_methodology/ nonexperimental.pdf.
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Pew Charitable Trusts, Philadelphia, PA.; Ambrose Monell Foundation, New York, NY.; Ford Foundation, New York, NY.; Grable Foundation, Pittsburgh, PA.; Surdna Foundation, Inc., New York, NY.; Administration for Children and Families (DHHS), Washington, DC.; Alcoa Foundation, Pittsburgh, PA.; Fannie Mae Foundation, Washington, DC.; George Gund Foundation, Cleveland, OH.; New York Times Co., NY.; Open Society Inst., New York, NY.
Authoring Institution: Manpower Demonstration Research Corp., New York, NY.