Educational measurement and evaluation experts generally agree that increasing stakeholders' assessment literacy will yield a variety of positive benefits, especially broadening the range of assessment formats teachers use to measure students' mastery of high level, more cognitively complex learning outcomes. But in the context of education accountability as currently structured in American schools, such efforts also may lead teachers to become more sophisticated in test preparation activities and to narrow both their instruction and classroom assessment practices specifically to enhance students' performance on prescribed, annual high-stakes accountability assessments. This article explains why that is so, describes the process by which it occurred in one state, and offers specific suggestions as to how it might be avoided.