It is a matter of concern when large, federally funded programs are evaluated using designs that produce misleading information. In this article, we discuss problems associated with an evaluation design that was adopted by the U.S. Department of Education, Office of Special Education Programs (OSEP) to document the performance of a major early intervention (EI) program, serving young children with developmental delays and disabilities. In particular, we focus on OSEP's requirement that states use a single group pre--post comparison design to evaluate the impact of EI on child outcomes. We also provide a data-based illustration that shows this evaluation design cannot distinguish child progress that is due to EI services from changes associated with other factors, such as regression to the mean. We hope this work will support the adoption of evaluation designs that are more in line with accepted principles of program evaluation.