Federal school accountability policies like No Child Left Behind were based on a logic that measuring school performance and making the results public through tools like school report cards would incentivize educators to create strategies for improving school quality. Yet, most schools needed more than incentives to be able to design improvement strategies that would lead to all students becoming proficient in standard subjects like math and ELA. As a result, states and school districts implemented an infrastructure of supports. To date, there is little research that considers how support providers use accountability tools to diagnose problems and design targeted improvement strategies. Without better knowledge of how schools and providers commit to particular improvement strategies, it is difficult to determine whether we need better school report cards or strategies, or both to improve school quality. This study aims to address this gap by examining how four Children First Networks in New York City used accountability metrics to develop targeted improvement strategies, which led to distinctly different improvement strategies. The article closes with implications for policy.