In this article, we test an alternative approach to creating a counterfactual basis for estimating individual and average treatment effects. Instead of using control/comparison groups or before-measures, the so-called Counterfactual as Self-Estimated by Program Participants (CSEPP) relies on program participants' self-estimations of their own…

In evaluation and applied social research, focus groups may be used to gather different kinds of evidence (e.g., opinion, tacit knowledge). In this article, we argue that making focus group design choices explicitly in relation to the type of evidence required would enhance the empirical value and rigor associated with focus group utilization. We…

This article proposes an innovative approach to estimating the counterfactual without the necessity of generating information from either a control group or a before-measure. Building on the idea that program participants are capable of estimating the hypothetical state they would be in had they not participated, the basics of the Roy-Rubin model…

Developmental evaluation (DE), essentially conceptualized by Patton over the past 30 years, is a promising evaluative approach intended to support social innovation and the deployment of complex interventions. Its use is often justified by the complex nature of the interventions being evaluated and the need to produce useful results in real time.…

The growth in the availability of longitudinal data--data collected over time on the same individuals--as part of program evaluations has opened up exciting possibilities for evaluators to ask more nuanced questions about how individuals' outcomes change over time. However, in order to leverage longitudinal data to glean these important…

In this paper, the author argues that evaluation theory and practice interact insufficiently today, even though early evaluation theorists expected them to be closely intertwined. She views this limited connection as the result of differing interests on the part of theorists and practitioners, differing frequencies of dissemination, and differing…

Sound evaluation planning requires numerous decisions about how constructs in a program theory will be translated into measures and instruments that produce evaluation data. This article, the first in a dialogue exchange, examines how decisions about measurement are (and should be) made, especially in the context of small-scale local program…

Attention to evaluation quality is commonplace, even if sometimes implicit. Drawing on her 2010 Presidential Address to the American Evaluation Association, Leslie Cooksy suggests that evaluation quality depends, at least in part, on the intersection of three factors: (a) evaluator competency, (b) aspects of the evaluation environment or context,…

This article provides a frame for evaluation of natural resource interventions, which necessarily involves both human and natural systems. Two-system evaluands require us to adapt evaluation methods for comparison and attribution and to address differences in time and space occurring across the systems as well as potentially very different values…

Where do evaluators find resources on ethics and ethical practice? This article highlights a relatively new online resource, a centerpiece project of the National Center for Professional and Research Ethics (NCPRE), which brings together information on best practices in ethics in research, academia, and business in an online portal and center. It…

The examination of cross-classified category data is common in evaluation and research, with Karl Pearson's family of chi-square tests representing one of the most utilized statistical analyses for answering questions about the association or difference between categorical variables. Unfortunately, these tests are also among the more commonly…

Although the general conceptual basis appeared far earlier, theory-driven evaluation came to prominence only a few decades ago with the appearance of Chen's 1990 book "Theory-Driven Evaluations." Since that time, the approach has attracted many supporters as well as detractors. In this paper, 45 cases of theory-driven evaluations, published over a…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

In the 1960s, evaluation thrived in the sunshine of political support for our work. We also experienced the dark side of the force of politics: efforts to introduce partisan biases favorable to special interests. In response to politics as contamination, at least two families of stances developed: populist and public interest. The populist stances…

2011 is the 25th anniversary of the American Evaluation Association. Such occasions are often recognized both by looking back to assess and celebrate accomplishments of the past and by looking ahead to anticipate the problems and potential of the future. In the past 25 years, the profession of evaluation has experienced rapid, ubiquitous growth,…