Although collaboration is recognized as an effective means to address multifaceted community issues, successful collaboration is difficult to achieve and failure is prevalent. To effectively collaborate, collaborators must recognize the strengths and weaknesses within their own efforts. Using Mattessich and colleagues' work as a springboard,…

Qualitative data offer advantages to evaluators, including rich information about stakeholders' perspectives and experiences. However, qualitative data analysis is labor-intensive and slow, conflicting with evaluators' needs to provide punctual feedback to their clients. In this method note, we contribute to the literature on rapid…

In this article, we test an alternative approach to creating a counterfactual basis for estimating individual and average treatment effects. Instead of using control/comparison groups or before-measures, the so-called Counterfactual as Self-Estimated by Program Participants (CSEPP) relies on program participants' self-estimations of their own…

Considerable empirical research, along with a growing body of conceptual and theoretical literature, exists on the role of culture and context in evaluation. Less scholarship has examined culturally responsive surveys in the context of international evaluation. In this article, the authors present lessons learned from the development,…

Researchers have become increasingly interested in programs' main and interaction effects of two variables (A and B, e.g., two treatment variables or one treatment variable and one moderator) on outcomes. A challenge for estimating main and interaction effects is to eliminate selection bias across A-by-B groups. I introduce Rubin's…

Nonprofit organizations are under great pressure to use evaluations to show that their programs "work" and that they are "effective." However, empirical evidence indicates that nonprofits struggle to perform useful evaluations, especially when conducted under accountability pressures. An increasing body of evidence highlights…

Despite the claimed success of the 1996 Welfare Reform, little research using multivariate regression has examined changes in multiple public safety-net programs. Thus, we still do not know whether public safety-net programs for the poor have shrunk or increased nationwide, along with the sharp declines in cash assistance. Using state-level data…

Predictors of retention of participants in a longitudinal study and heterogeneity between communities were investigated using a multilevel logistic regression model. Data from the longitudinal outcome study of the national evaluation of the Comprehensive Community Mental Health Services for Children and Their Families program and information on…

Process use refers to the ways in which stakeholders and/or evaluands change as a function of participating in evaluation activities. Although the concept of process use has been well discussed in the literature, exploration of methodological strategies for the measurement and assessment of process use has been limited. Typically, empirical…

This classification study examines evaluators' coursework training as a way of understanding evaluation practice. Data regarding courses that span methods and evaluation topics were collected from evaluation practitioners. Using latent class analysis, we establish four distinct classes of evaluator course-taking patterns: quantitative,…

Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design's validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the…

In evaluation and applied social research, focus groups may be used to gather different kinds of evidence (e.g., opinion, tacit knowledge). In this article, we argue that making focus group design choices explicitly in relation to the type of evidence required would enhance the empirical value and rigor associated with focus group utilization. We…

Extracted from a larger study of the educational evaluation profession, this qualitative analysis explores how evaluator identity is shaped with constant reference to political economy, knowledge work, and personal history. Interviews with 24 social scientists who conduct or have conducted evaluations as a major part of their careers examined how…

Quality training opportunities for evaluators will always be important to the evaluation profession. While studies have documented the number of university programs providing evaluation training, additional information is needed concerning what content is being taught in current evaluation courses. This article summarizes the findings of a survey…

Evaluators widely agree that stakeholder involvement is a central aspect of effective program evaluation. With the exception of articles on collaborative evaluation approaches, however, a systematic review of the breadth and depth of the literature on stakeholder involvement has not been published. In this study, we examine peer-reviewed empirical…