Standard setting is a systematic process that uses a combination of judgmental and empirical procedures to make recommendations about where on the score continuum "cut scores" should be placed. Cut scores divide the score scale into categories consistent with the descriptions of student performance associated with multiple levels of achievement. Critical decisions will be based on interpretations emerging from how students are categorized into each level. Ensuring that the methodology used to set the cut scores is sound and defensible is a prerequisite for valid interpretations of student performance. Innovations in standard setting methodology in the first decade of this century have been prolific; the number of articles, chapters and presentations are too numerous to cite or review here, but the fact that there have been at least three books just on standard setting this decade is testimony to the amount of research and development in this area (Cizek, 2001; Cizek & Bunch, 2007; Zieky, Perie, & Livingston, 2008). In this paper, the authors question whether all of this work, effort, and resources has been misdirected given the less-than-adequate work on one of the most critical inputs to the standard setting process: the achievement level descriptions. In other words, like the crude adage "garbage in, garbage out," innovations in standard setting methodology are attenuated until some necessary preconditions are met. The authors outline these necessary preconditions, their promise for improving standard setting, their promise for improving the accountability system and the reform effort for which standard setting is a means, and suggested research to make progress on those necessary preconditions. (Contains 2 tables, 1 figure and 7 footnotes.)