In this article, we test an alternative approach to creating a counterfactual basis for estimating individual and average treatment effects. Instead of using control/comparison groups or before-measures, the so-called Counterfactual as Self-Estimated by Program Participants (CSEPP) relies on program participants' self-estimations of their own…

This article proposes an innovative approach to estimating the counterfactual without the necessity of generating information from either a control group or a before-measure. Building on the idea that program participants are capable of estimating the hypothetical state they would be in had they not participated, the basics of the Roy-Rubin model…

Developmental evaluation (DE), essentially conceptualized by Patton over the past 30 years, is a promising evaluative approach intended to support social innovation and the deployment of complex interventions. Its use is often justified by the complex nature of the interventions being evaluated and the need to produce useful results in real time.…

Longitudinal substance abuse research has often been compromised by high rates of attrition, thought to be the result of the lifestyle that often accompanies addiction. Several studies have used strategies including collection of locator information at the baseline assessment, verification of the information, and interim contacts prior to…

This article synthesizes interview data from evaluation directors and top executives of philanthropic foundations on how evaluation might better advance their missions. In key informant interviews, respondents commented on the purposes of evaluation from the foundation's perspective, challenges to effective evaluation, and the means by which…

Using exogenous characteristics to identify endogenous subgroups, the approach discussed in this method note creates symmetric subsets within treatment and control groups, allowing the analysis to take advantage of an experimental design. In order to maintain treatment--control symmetry, however, prior work has posited that it is necessary to use…

There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called "between-studies," do not make direct…

Sweden has a low prevalence of bullying and Swedish schools are legally obliged to have anti-bullying policies. Many commercial programs are available. A mixed methods, quasi-experimental, concurrent evaluation of 8 programs, chosen from a pool of 21 widely used anti-bullying programs, was planned. Preliminary data, based on 835 stakeholder…

In recent years, increased focus on the effectiveness and accountability of prevention and intervention programs has led to greater government funding for the implementation and spread of evidence-based health and human service delivery models. In particular, attention has been paid to programs that require significant infrastructure investment…

Researchers and policy makers are increasingly dissatisfied with the "average treatment effect." Not only are they interested in learning about the overall causal effects of policy interventions, but they want to know what specifically it is about the intervention that is responsible for any observed effects. In the U.S., using…

Linking evaluation methods to the several phases of a program's life cycle can provide evaluation planners and funders with guidance about what types of evaluation are most appropriate over the trajectory of social and educational programs and other interventions. If methods are matched to the needs of program phases, evaluation can and should…

In attempting to use a realistic evaluation approach to explore the role of Community Parents in early parenting programs in Toronto, a novel technique was developed to analyze the links between contexts (C), mechanisms (M) and outcomes (O) directly from experienced practitioner interviews. Rather than coding the interviews into themes in terms of…

This article provides a frame for evaluation of natural resource interventions, which necessarily involves both human and natural systems. Two-system evaluands require us to adapt evaluation methods for comparison and attribution and to address differences in time and space occurring across the systems as well as potentially very different values…

Health promotion and social betterment program interventions are based on either formal theory from academia or stakeholder theory from stakeholders' observations and experiences in working with clients. Over time, formal theory-based interventions have acquired high prestige, while stakeholder theory-based interventions have been held in low…

Multilevel modeling techniques facilitated examination of relationships between fidelity indicators and outcomes associated with a summer literacy intervention. Three-level growth models were specified to capture the extent to which students experienced instruction and to demonstrate the ways in which dosage-response relationships manifest in…