Shadowing is a data collection method that involves following a person, as they carry out those everyday activities relevant to a research study. This article explores the use of shadowing in a formative evaluation of a professional development school (PDS). Specifically, this article discusses how shadowing was used to understand the role of a…

Performance-based accountability along with budget tightening has increased pressure on publicly funded organizations to develop and deliver programs that produce meaningful social benefits. As a result, there is increasing need to undertake formative evaluations that estimate preliminary program outcomes and identify promising program components…

When evaluators plan site-randomized experiments, they must conduct the appropriate statistical power analyses. These analyses are most likely to be valid when they are based on data from the jurisdictions in which the studies are to be conducted. In this method note, we provide software code, in the form of a SAS macro, for producing statistical…

Multilevel modeling techniques facilitated examination of relationships between fidelity indicators and outcomes associated with a summer literacy intervention. Three-level growth models were specified to capture the extent to which students experienced instruction and to demonstrate the ways in which dosage-response relationships manifest in…

There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and…

This is the first examination of exemplary evaluation under a new editorial approach, in which the authors are attempting not only to report how the evaluation was conducted and to explain the rationale for design and implementation but also to examine the conditions, events, or actions that might have contributed to its exemplary status. This…

Most K-12 evaluations are designed to make inferences about how a program implemented at the classroom or school level affects student learning outcomes and such inferences inherently involve hierarchical data structure. One methodological challenge for evaluators is linking program implementation factors typically measured at the classroom or…

It has been acknowledged for several decades that programs interact with context. The nature of this interactivity, and how it defines a program, has not been adequately addressed. We view this lacuna as a function of the dominant theoretical perspectives guiding knowledge of program operations. We propose the actor-network theory (ANT) and its…

This article describes an educational program evaluation conducted by Dr. Eric Barela. Barela is a senior educational research analyst with the research and planning division of the Los Angeles Unified School District (LAUSD). He joined the division as an urban education research fellow and has been an internal evaluator with LAUSD for almost six…

This article uses a policy scientific approach to reconstruct assumptions underlying the Dutch Educational Supervision Act. We show an example of how to reconstruct and evaluate a program theory that is based on legislation of inspection. The assumptions explain how inspection leads to school improvement. Evaluation of these assumptions is used to…