NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ689219
Record Type: Journal
Publication Date: 2004
Pages: 33
Abstractor: Author
ISBN: N/A
ISSN: ISSN-1098-2140
EISSN: N/A
Shoestring Evaluation: Designing Impact Evaluations under Budget, Time and Data Constraints
Bamberger, Michael; Rugh, Jim; Church, Mary; Fort, Lucia
American Journal of Evaluation, v25 n1 p5-37 2004
The paper discusses two common scenarios in which evaluators must conduct impact evaluations when working under budget, time, or data constraints. Under the first scenario the evaluator is not called in until the project is already well advanced, and there is a tight deadline for completing the evaluation, frequently combined with a limited budget and without access to baseline data. Under the second scenario the evaluator is called in early, but for budget, political, or methodological reasons it is not possible to collect baseline data on a control group and sometimes not even on the project population. As a result of these constraints, many of the basic principles of impact evaluation design (comparable pretest-posttest design, control group, instrument development and testing, random sample selection, control for researcher bias, thorough documentation of the evaluation methodology, etc.) are often sacrificed. We describe the "Shoestring Evaluation" approach which is being developed to provide tools for ensuring the highest quality evaluation possible under constraints of limited budget, time and data availability. While most of the data collection and analysis techniques will be familiar to experienced evaluators, what is new is the combination of these techniques into an integrated six-step approach which covers: (1) planning and scoping the evaluation, (2-4) options for dealing with constraints related to costs, time and data availability (which could include reconstructing baseline conditions and control groups), (5) identifying the strengths and weaknesses (threats to validity and adequacy) of the evaluation design, and (6) taking measures to address the threats and strengthen the evaluation design and conclusions. When necessary, many of these corrective measures can be introduced at a very late stage, even when the draft evaluation report has already been produced. Although the approach was designed to assist evaluators working in developing countries, the principles are equally applicable for conducting evaluations under similar constraints in industrial nations. Case studies are included to illustrate the different Shoestring approaches, drawing from the authors' experience in Eastern Europe, Africa, Asia, and Latin America.
Sage Publications, 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243 (Toll Free); Fax: 800-583-2665 (Toll Free).
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A