The scoring process is critical in the validation of tests that rely on constructed responses. Documenting that readers carry out the scoring in ways consistent with the construct and measurement goals is an important aspect of score validity. In this article, rater cognition is approached as a source of support for a validity argument for scores…

The engagement of teachers as raters to score constructed response items on assessments of student learning is widely claimed to be a valuable vehicle for professional development. This paper examines the evidence behind those claims from several sources, including research and reports over the past two decades, information from a dozen state…

In a recent article published in "EM:IP," Kingston and Nash report on the results of a meta-analysis on the efficacy of formative assessment. They conclude that the average effect of formative assessment on student achievement is about 0.20 SD units. This would seem to dispel the myth that effects between 0.40 and 0.70 can be attributed to…

The purpose of this study was to investigate the connotation of performance labels used in standard setting. For example, do the performance labels "basic," "proficient," and "advanced" hold different connotations than "limited knowledge," "satisfactory," and "distinguished"? If these terms hold different connotations, such differences may play a…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Postsecondary schools have traditionally relied on admissions tests such as the SAT and ACT to select students. With high school achievement assessments in place in many states, it is important to ascertain whether scores from those exams can either supplement or supplant conventional admissions tests. In this study we examined whether the Arizona…

Assessments labeled as formative have been offered as a means to improve student achievement. But labels can be a powerful way to miscommunicate. For an assessment use to be appropriately labeled "formative," both empirical evidence and reasoned arguments must be offered to support the claim that improvements in student achievement can be linked…

This article describes an alignment procedure, called Links for Academic Learning (LAL), for examining the degree of alignment of alternate assessments based on alternate achievement standards (AA-AAS) to grade-level content standards and instruction. Although some of the alignment criteria are similar to those used in general education…

The Growth Model Pilot Program (GMPP) was introduced as a response to criticism of the reliance of No Child Left Behind on status-based indicators. Although incorporating a growth component appears to be a step in the right direction, it adds a level of complexity that brings to the fore new concerns. Three of the four papers in this special issue…

A key intent of the NCLB growth pilot is to reward low-status schools who are closing the gap to proficiency. In this article, we demonstrate that the capability of proposed models to identify those schools depends on how the growth model is incorporated into accountability decisions. Six pilot-approved growth models were applied to vertically…

Annual student achievement data derived from state assessment programs have led to widespread enthusiasm for statistical models suitable for longitudinal analysis. The current policy environment's adherence to high stakes accountability vis-a-vis No Child Left Behind (NCLB)'s universal proficiency mandate has fostered an impoverished view of what…

Most growth models implicitly assume that test scores have been vertically scaled. What may not be widely appreciated are the different choices that must be made when creating a vertical score scale. In this paper empirical patterns of growth in student achievement are compared as a function of different approaches to creating a vertical scale.…

States participating in the Growth Model Pilot Program reference individual student growth against "proficiency" cut scores that conform with the original No Child Left Behind Act (NCLB). Although achievement results from conventional NCLB models are also cut-score dependent, the functional relationships between cut-score location and growth…

Examined in this study were the effects of reducing anchor test length on student proficiency rates for 12 multiple-choice tests administered in an annual, large-scale, high-stakes assessment. The anchor tests contained 15 items, 10 items, or five items. Five content representative samples of items were drawn at each anchor test length from a…

This paper describes a comprehensive model of alignment that provides a foundation for meaningful reporting of students' academic progress over time. The model includes both horizontal and vertical alignment as integral parts of the development of content standards, test blueprints, items, item pools, instruments, performance level descriptors,…