We contribute to debate about causal inferences in educational research in two ways. First, we quantify how much bias there must be in an estimate to invalidate an inference. Second, we utilize Rubin's causal model to interpret the bias necessary to invalidate an inference in terms of sample replacement. We apply our analysis to an inference…

Drawing on our five years of experience developing WWC evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues and common pitfalls in designing and conducting impact studies in education, and provide practical…

The influence of teachers' qualifications on their teaching practice has been subject to debate. Literature reviews do not settle these debates, partly because the literature is uneven and partly because reviews capture only narrow slices of literature. In particular, many reviews eliminate qualitative studies. Yet without examining qualitative…

The higher education diversity programs that Texas enacted after "Hopwood v. University of Texas" banned affirmative action had unexpected positive consequences for the state's high schools. The Texas top 10% law, the Longhorn Opportunity Scholarship and Century Scholarship programs, and the Towards Excellence, Access and Success Grant program…

Interest has rapidly increased in studies that randomly assign classrooms or schools to interventions. When well implemented, such studies eliminate selection bias, providing strong evidence about the impact of the interventions. However, unless expected impacts are large, the number of units to be randomized needs to be quite large to achieve…

Experiments that assign intact groups to treatment conditions are increasingly common in social research. In educational research, the groups assigned are often schools. The design of group-randomized experiments requires knowledge of the intraclass correlation structure to compute statistical power and sample sizes required to achieve adequate…

This article addresses conflicting statements about the impact of accountability policies in which some argue that testing undermines good teaching, while others claim that it stimulates improvement. The authors begin with the assumption that it is important to explore implementers' cognitive perspectives in order to understand a policy's effects.…

Improving the validity and reliability of surveys is a critical part of the response to the call for improved rigor of education research, policy analysis and evaluation. Too often we create inquiry tools without validating our measures against how respondents interpret our questions, and therefore collect data of questionable quality. The purpose…

In this paper, the authors present a complex set of interactions, agents, pressures, and contextual factors that intersect during the earliest years of major effort at education policy reform. While the focus of this analysis is exclusively on state policy changes and school district response in New York, the issues involved are all but universal…

Numerous researchers have speculated on and attempted to quantify the impact of policies designed to increase school choice and competition between schools, and they have reached widely differing conclusions. In this article, we provide an overview of the research that focuses on the potential achievement effects of greater private sector…

Drawing inspiration from Shulman's (1986) construct of pedagogical content knowledge, we propose that leadership content knowledge is a missing paradigm in the analysis of school and district leadership. After defining leadership content knowledge as that knowledge of academic subjects that is used by administrators when they function as…

Many researchers who study the relations between school resources and student achievement have worked from a causal model, which typically is implicit. In this model, some resource or set of resources is the causal variable and student achievement is the outcome. In a few recent, more nuanced versions, resource effects depend on intervening…

This article features case studies of two elementary schools, identified as exemplary by Washington educators, as they worked to enact Washington's Essential Academic Learning Requirements and help students achieve proficiency on the Washington Assessment of Student Learning. We describe school- and classroom-level practices and analyze the…

Studying change in student achievement is of central importance in numerous areas of educational research, including efforts to monitor school performance, investigations of the effects of educational interventions over time, and school effects studies focusing on how differences in school policies and practices relate to differences in student…

We use several national surveys to provide a 50-year perspective on time spent on homework. The great majority of American children at all grade levels now spend less than one hour studying on a typical day--an amount that has not changed substantially in at least 20 years. Moreover, high school students in the late 1940s and early 1950s studied…