Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

Evaluators widely agree that stakeholder involvement is a central aspect of effective program evaluation. With the exception of articles on collaborative evaluation approaches, however, a systematic review of the breadth and depth of the literature on stakeholder involvement has not been published. In this study, we examine peer-reviewed empirical…

This article describes the process by which we created a recruitment protocol for engaging adolescent sexual assault victims in a qualitative evaluation study. Working in collaboration with forensic nurses, rape victim advocates, adolescent rape survivors, and our institutional review board (IRB), we created a prospective recruitment method…

The growth in the availability of longitudinal data--data collected over time on the same individuals--as part of program evaluations has opened up exciting possibilities for evaluators to ask more nuanced questions about how individuals' outcomes change over time. However, in order to leverage longitudinal data to glean these important…

Shadowing is a data collection method that involves following a person, as they carry out those everyday activities relevant to a research study. This article explores the use of shadowing in a formative evaluation of a professional development school (PDS). Specifically, this article discusses how shadowing was used to understand the role of a…

This article describes lessons learned about implementing evaluations in hospital settings. In order to overcome the methodological dilemmas inherent in this environment, we used a practical participatory evaluation (P-PE) strategy to engage as many stakeholders as possible in the process of evaluating a clinical demonstration project.…

In this commentary, we examine Braverman's insights into the trade-offs between feasibility and rigor in evaluation measures and reject his assessment of the trade-off as a zero-sum game. We, argue that feasibility and policy salience are, like reliability and validity, intrinsic to the definition of a good measure. To reduce the tension between…

Evidence-based insight on the effectiveness of reminders in web-based data collection in online panels is scarce. Thirty-eight studies were conducted in three different online panels to examine the effect of reminders on response rates, retention rates, and two facets of response quality (i.e., item omissions and response nondifferentiation). The…

Major advances in the number, capabilities, and quality of state, national, and transnational databases have opened up new opportunities for evaluators. Both large-scale data sets collected for administrative purposes and those collected by other researchers can provide data for a variety of evaluation-related activities. These include (a)…

To describe the recent practice of evaluation, specifically method and design choices, the authors performed a content analysis on 117 evaluation studies published in eight North American evaluation-focused journals for a 3-year period (2004-2006). The authors chose this time span because it follows the scientifically based research (SBR)…

In recent years, there has been increasing attention to the importance of interagency collaboration for improving community well-being, environmental and public health, and educational outcomes. This article uses a mixed-methods approach including network analysis to examine the changes in interagency collaboration in one site funded by the Safe…

The use of theory-driven evaluation is an emerging practice in the military--an aspect generally unknown in the civilian evaluation community. First developed during the 1991 Gulf War and applied in both the Balkans and Afghanistan, these techniques are now being examined in the North Atlantic Treaty Organisation (NATO) as a means to evaluate the…

This project integrates elements of responsive evaluation and participatory evaluation to compare three evaluation data collection methods for use with a hard-to-find (HTF), traumatized, vulnerable population: rape victims seeking postassault medical forensic care. The first method involves on-site, in-person data collection, immediately…

Although random assignment is generally the preferred methodology in impact evaluations, it raises numerous ethical concerns, some of which are addressed by securing participants' informed consent. However, there has been little investigation of how consent is obtained in social experiments and the amount of information that can be conveyed--and…

Increasingly, government, foundations, and others are asking community-based organizations for more evaluation information. Although the demand for this information may be increasing, the field knows very little about the way organizations are responding to this demand. Using data collected from interviews and a mail survey of community-based…