Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

Developmental evaluation (DE), essentially conceptualized by Patton over the past 30 years, is a promising evaluative approach intended to support social innovation and the deployment of complex interventions. Its use is often justified by the complex nature of the interventions being evaluated and the need to produce useful results in real time.…

In this Exemplars case, the fifth and final under the direction of the current coeditors, the authors present a reflective account of an ongoing, complex, multiyear, multinational monitoring and evaluation (M&E) system conducted for African Women in Agricultural Research and Development (AWARD), an international development program. The…

Tobacco use continues to be the leading preventable cause of mortality and morbidity in North America and Quitlines are one of the primary cessation resources available to assist tobacco users with quitting. Implementation of best practices is important to the success of quitlines, but unfortunately, it is a complex and elusive process often…

In recent years, increased focus on the effectiveness and accountability of prevention and intervention programs has led to greater government funding for the implementation and spread of evidence-based health and human service delivery models. In particular, attention has been paid to programs that require significant infrastructure investment…

Multilevel modeling techniques facilitated examination of relationships between fidelity indicators and outcomes associated with a summer literacy intervention. Three-level growth models were specified to capture the extent to which students experienced instruction and to demonstrate the ways in which dosage-response relationships manifest in…

I define a treatment effect in terms of a comparison of outcomes and provide a typology of all possible comparisons that can be used to estimate treatment effects, including comparisons that are relatively unknown in both the literature and practice. I then assess the relative merit, worth, and value of all possible comparisons based on the…

A theory-driven approach to evaluation (TDE) emphasizes the development and empirical testing of conceptual models to understand the processes and mechanisms through which programs achieve their intended goals. However, most reported applications of TDE are limited to large-scale experimental/quasi-experimental program evaluation designs. Very few…

The authors present a case study examining the potential for policies to be "evidence-based." To what extent is it possible to say that a decision to implement a complex social intervention is warranted on the basis of available empirical data? The case chosen is whether there is sufficient evidence to justify banning smoking in cars carrying…

In this article, we explore the evaluation capacity of today's nonprofit organizations. We report the findings of a cluster analysis that suggest that when it comes to evaluation, there are three types of nonprofit organizations. The first type of nonprofit organization is one that, by most accounts, is satisfied with their evaluation efforts.…

There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and…

Environment and sustainable development show how policies are becoming ever more complex and ambiguous. This trend calls for new evaluation approaches. They need to be more clearly focused on specific, explicit concerns. They must be driven by a strategic concept of use to overcome the vulnerability to manipulation of many integrative, essentially…

Networks aiming for fundamental changes bring together a variety of actors who are part and parcel of a problematic context. These system innovation projects need to be accompanied by a monitoring and evaluation approach that supports and maintains reflexivity to be able to deal with uncertainties and conflicts while challenging current practices…

Most K-12 evaluations are designed to make inferences about how a program implemented at the classroom or school level affects student learning outcomes and such inferences inherently involve hierarchical data structure. One methodological challenge for evaluators is linking program implementation factors typically measured at the classroom or…

It has been acknowledged for several decades that programs interact with context. The nature of this interactivity, and how it defines a program, has not been adequately addressed. We view this lacuna as a function of the dominant theoretical perspectives guiding knowledge of program operations. We propose the actor-network theory (ANT) and its…