The growth in the availability of longitudinal data--data collected over time on the same individuals--as part of program evaluations has opened up exciting possibilities for evaluators to ask more nuanced questions about how individuals' outcomes change over time. However, in order to leverage longitudinal data to glean these important…

A total of 271 early childhood professionals completed pre- and post training knowledge assessments in True-False only (TF) or True-False with "unsure" option formats (TFU). In Study 1, only TFU format was used. In Study 2, participants were randomly assigned to TF or TFU formats. Responses which were initially "unsure" were more likely than…

Today's public policy discussions increasingly focus on how "networks" of public and private actors collaborate across organizational, sectoral, and geographical boundaries to solve increasingly complex problems. Yet, many of evaluation's key concepts, including the evaluator's role, assume an evaluand that is programmatically or organizationally…

It is familiar in the world of systems building that funders start out saying they are not particularly interested in attribution, only to later begin using words like "prove" and "impact." Boards begin by asking for intelligent reflection from the 30,000-foot level and then suddenly want to know how many more children are ready for school.…

In April 2007, more than 50 practitioners, advocates, funders, and evaluators met for 2 days in Pittsburgh, Pennsylvania, to discuss the issues that arise in evaluating systems change or systems-building initiatives and to suggest approaches to resolving those issues. The symposium was designed to be as realistic and practical as possible by…

Are the five states that are participating in the Great Start project, in fact, off to a Great Start? The simple answer from a casual read of this excellent case study appears to be "yes." Any early learning initiative that begins with a scientific evaluation is off to a great start. Sadly, some public and private funders have begun early learning…

In April 2007, more than 60 people--practitioners, advocates, funders, and evaluators--met for 2 days in Pittsburgh, Pennsylvania, to discuss the challenges that arise in evaluating systems change or systems-building initiatives and to suggest approaches to resolving those challenges. The symposium was convened by the partners of BUILD, a…

The Case of Top Beginnings and the Missing Child Outcomes is a fictionalized case study of a set of evaluation challenges faced in evaluating comprehensive initiatives that are seeking to build systems. In this case, the issue is a multistate foundation-funded early learning system building initiative that has to date primarily employed a case…

The evaluation of an intervention delivered across multiple treatment sites presents a unique opportunity for evaluators to gauge the manner and degree to which the "impact" of treatment varies across implementation conditions and different target populations. However, the availability of implementation data for each treatment site, while…

Cost studies are program evaluations that judge program worth by relating program costs to program benefits. There are three sets of strategies: cost-benefit, cost-effectiveness, and cost-utility analysis, although the last appears infrequently. The authors searched relevant databases to identify 103 cost studies in education and then reduced the…

This study investigates how program evaluation has contributed to the definition of quality in early childhood programs and what social and cultural conditions have influenced these definitions. First, a review of evaluation reports from the last 3 decades identified three types of quality used to evaluate early childhood programs: (a)…