In many large-scale assessment programs, achievement level descriptors (ALDs) provide a critical role in communicating what scores on the assessment mean and in interpreting what examinees know and are able to do based on their test performance. Based on their test performance, examinees are often classified into performance categories. The…

Two concerns related to setting performance standards on educational assessments are discussed. First, criteria for a standard-setting process from the point of view of a standard-setting sponsor, called here "institutional criteria," are developed using a state department of education as an example. Four institutional criteria are proposed: (a)…

Standard-setting research has relied very heavily on certain types of information that, although helpful, may be inadequate for judging the quality of standard-setting results. To increase confidence in standard-setting results, the measurement community must understand far more about the panelists' task and about the factors that influence their…

In a coherent educational assessment system, the relation between the cut scores across the grades of a content area should reflect the goals of the educational system moderated by the state of the standards within that system. In this article, we discuss the need for consistent cut scores across the grades of a content area; we present and…

Under the adequate yearly progress requirements of the No Child Left Behind (NCLB) Act (2001), states are currently faced with the challenge of demonstrating continuous improvement in student performance in reading and mathematics. Beginning in 2007 to 2008, science will be required as a component of the NCLB Act. This article describes South…

The National Assessment Governing Board used a new method to set achievement level standards on the 2005 Grade 12 NAEP Math test. In this article, we summarize our independent evaluation of the process used to set these standards. The evaluation data included observations of the standard-setting meeting, observations of advisory committee meetings…

Only a small number of qualitative studies have investigated panelists' experiences during standard-setting activities or the thought processes associated with panelists' actions. This qualitative study involved an examination of the experiences of 11 panelists who participated in a prior, one-day standard-setting meeting in which either the…

Panelists in an operational standard-setting study were asked to share their thoughts in written form at important points in the process itself--before the meeting started, after training, after completing the 1st and 2nd sets of ratings, immediately following the discussion between Rounds 1 and 2, and so on. The item mapping method of Plake and…

This article reviews the empirical literature on 9 topics about the modified Angoff standard-setting method that have been studied repeatedly in the literature, while taking into consideration the methodological warrant for the findings on the topics. It concludes that we can be reasonably confident about selecting the appropriate number of judges…

A brief summary of standard setting knowledge is presented, derived from about 20 methods that utilize a judgmental review process, the approach most relevant to the standard-setting strategies proposed in this special issue. Criteria for judging effectiveness and critiques of the methods discussed in the issue are offered. (SLD)

The process used to identify the provisional locations of points that would inform the National Assessment Governing Board in setting achievement levels for the science National Assessment of Educational Progress (NAEP) is described. This process includes identification of panelists for achievement-level setting and the method for converting…

This article addresses the set of articles in this special issue of "Applied Measurement in Education" and reflects on the issues that underlie the articles. The authors, as a set, represent many of the professionals who have developed and studied the methodological procedures related to instruction-assessment alignment over the past decade.…