The laudatory goal of evaluation capacity building is quite timely, given the spotlight on the potential use of program evaluation in the current environment which emphasizes performance reporting and evidence-based policy. A question that quickly comes to mind is: Where do you start? A seemingly reasonable starting point is to assess…

Listservs have become a popular computer-mediated method for communicating with others who share a common interest. They offer subscribers quick feedback and provide a forum for ideas and beliefs to be exchanged, discussed, and modified. Although rarely studied, in some practical fields listserves have been shown to increase communication of best…

Can this evaluation be saved? More precisely, can this evaluation be saved in such a way that both evaluator and client feel satisfied that their points of view were respected and both agree that the evaluation itself provides valid information obtained in a principled manner? Because the scenario describes a preliminary discussion and no contract…

The agency director in this case clearly does not value having a client representative on the evaluation steering committee, and it would appear that she has no experience working with the mentally ill in a collaborative fashion. The director not only believes that her clients are impaired and therefore incapable of contributing to the work of the…

In early 2002, Jean King, Mel Mark, Robin Miller, and Stacey Stockdill began a project to conduct oral history interviews with individuals who have made signal contributions to the program evaluation field and those well-placed observers who were present at and played a role in pivotal moments in the field. In developing this project, it is our…

This paper provides the results of a literature review on the use of practical, hands-on training experiences in evaluation course work and training programs. The review spans the years 1965-2003. I identified 18 articles that encompass four basic approaches for practical evaluation training: simulation, role-play, single course projects, and…

In this article, I outline how, with the appropriate mix of external requirements and internal motivation, structure and capacity, a school can promote and maintain a culture of evaluative practices, specifically data use, for continuous improvement. I draw upon qualitative data conducted in a study of an Edison Project school. The findings from…

This paper proposes a blended experimental and quasi-experimental research strategy which we believe will help improve the external validity of evaluations of welfare reform and other social policies. We draw upon data from New Jersey's imposition of a family cap, where welfare benefits instead of increasing remained the same for women who…

Despite evidence that the choice of dependent measures can significantly influence design sensitivity, many evaluators default to traditional measures that may be insensitive to intervention effects. This paper describes an innovative set of test development guidelines designed to select items and create aggregate scales that are better able to…

In the UK, following the Crime and Disorder Act of 1998, the Youth Justice Board (YJB) funded over 450 separate crime prevention schemes. The YJB in approving these various schemes stressed the importance of monitoring and evaluation. Local evaluators were appointed whose role was to report back to the funding agency and the national evaluators.…

"Ethical Challenges," in my experience, is invariably interesting, often instructive and sometimes amusing. Some of the most engaging stimulus scenarios raise thorny evaluation practice issues that ultimately lead to disparate points of view about the nature of the issue and how to handle it (Datta, 2002; Smith, 2002). Despite my poor performance…

The passage of the Contract with America Advancement Act terminated the Social Security Administrations Disability Insurance and Supplemental Security Income benefits for persons diagnosed with drug or alcohol addiction (also known as DA&A recipients), effective January 1, 1997. From an evaluation standpoint, the law created the opportunity…

Today, educational evaluation theory and practice face a critical juncture with the kind of educational accountability evaluation legislated by No Child Left Behind. While the goal of this kind of educational accountability is to improve education, it is characterized by a hierarchical, top-down approach to improving educational achievement…

Does taking advanced courses improve scores on basic achievement tests? This question is difficult to answer because there are many systematic differences between students who choose to take advanced courses and those who don't. This paper uses cutting-edge propensity score methods to control for the selection bias that exists when studying the…

Despite availability of empirically supported school-based substance abuse prevention programs, adoption and implementation fidelity of such programs appear to be low. A replicated case study was conducted to investigate school adoption and implementation processes of the EXSELS model (Project ALERT delivered by program leaders through Cooperative…