This article describes lessons learned about implementing evaluations in hospital settings. In order to overcome the methodological dilemmas inherent in this environment, we used a practical participatory evaluation (P-PE) strategy to engage as many stakeholders as possible in the process of evaluating a clinical demonstration project.…

This essay engages questions of evaluator role and indigenous peoples participation in evaluation within colonial and decolonization contexts. Specifically, I critique the Western emphasis on cultural competence and contrast the utility of "mainstream" evaluation approaches alongside three indigenous inquiry models (Te Kotahitanga,…

Organizational evaluation capacity building has been a topic of increasing interest in recent years. However, the actual dimensions of evaluation capacity have not been clearly articulated through empirical research. This study sought to address this gap by identifying the key dimensions of evaluation capacity in Canadian federal government…

In this article, Michael Quinn Patton pays tribute to Carol Hirschon Weiss, a woman who brought passion to evaluation use. She also brought attention to it, publishing the first article on evaluation in 1967. Weiss' many contributions have come into evaluation currency and overtaken formerly taken-for-granted assumptions. Indeed, her insights…

As the definition of organization has evolved, so have the approaches used to evaluate organizational performance. During the past 60 years, organizational theorists and management scholars have developed a comprehensive line of thinking with respect to organizational assessment that serves to inform and be informed by the evaluation discipline.…

This article explores the viability of online crowdsourcing for creating matched-comparison groups. This exploratory study compares survey results from a randomized control group to survey results from a matched-comparison group created from Amazon.com's MTurk crowdsourcing service to determine their comparability. Study findings indicate…

To answer "what works?" questions about policy interventions based on an experimental design, Peck (2003) proposes to use baseline characteristics to symmetrically divide treatment and control group members into subgroups defined by endogenously determined postrandom assignment events. Symmetric prediction of these subgroups in both…

Tobacco use continues to be the leading preventable cause of mortality and morbidity in North America and Quitlines are one of the primary cessation resources available to assist tobacco users with quitting. Implementation of best practices is important to the success of quitlines, but unfortunately, it is a complex and elusive process often…

In this commentary, we examine Braverman's insights into the trade-offs between feasibility and rigor in evaluation measures and reject his assessment of the trade-off as a zero-sum game. We, argue that feasibility and policy salience are, like reliability and validity, intrinsic to the definition of a good measure. To reduce the tension between…

Joseph S. Wholey is Professor Emeritus in the University of Southern California's (USC) School of Public Policy, Planning, and Development. He received his BA from Catholic University, Phi Beta Kappa and his MA in mathematics and PhD in philosophy from Harvard University. His work focuses on the use of strategic planning, performance measurement,…

Although evaluation theorists over the last two decades have argued for the importance of including stakeholders from marginalized groups in program planning and research, little is known about the degree of inclusion in program evaluation practice. In particular, we know little about the type and level of inclusion of people with intellectual,…

Evaluators' experiences with being pressured by a key stakeholder to misrepresent findings were investigated in an anonymous survey of a random sample of over 2500 American Evaluation Association members; a response rate of 37% was obtained. Overall, 42% of the respondents had encountered misrepresentation pressure, with 70% of this subgroup…

In Canada, in-depth evaluations of federal programs are intended to occur every 5 years. As such, evaluation is a periodic retrospective (lag) indicator examining results achieved versus program objectives. In a Canadian context, stand-alone evaluations have proved challenging to implement, time consuming, and not well adapted to annual management…

When planning and evaluating community-level initiatives focused on policy and environment change, it is useful to have estimates of the impact on behavioral outcomes of particular strategies (e.g., building a new walking trail to promote physical activity). We have created a measure of estimated strategy-level impact--"population dose"--based on…

In this article, we critique two recent theoretical developments about collaborative inquiry in evaluation--using logic models as a means to understand theory, and efforts to compartmentalize versions of collaborative inquiry into discrete genres--as a basis for considering future direction for the field. We argue that collaborative inquiry in…