With an increase in the number of online tests, the number of interruptions during testing due to unexpected technical issues seems to be on the rise. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees' scores. Researchers…

With an increase in the number of online tests, interruptions during testing due to unexpected technical issues seem unavoidable. For example, interruptions occurred during several recent state tests. When interruptions occur, it is important to determine the extent of their impact on the examinees' scores. There is a lack of research on this…

Compared to unidimensional item response models (IRMs), cognitive diagnostic models (CDMs) based on latent classes represent examinees' knowledge and item requirements using discrete structures. This study systematically examines the viability of retrofitting CDMs to IRM-based data with a linear attribute structure. The study utilizes a procedure…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

A potential undesirable effect of multistage testing is differential speededness, which happens if some of the test takers run out of time because they receive subtests with items that are more time intensive than others. This article shows how a probabilistic response-time model can be used for estimating differences in time intensities and speed…

Proposed two new indices to detect answer copying on a multiple choice test and conducted a simulation study to investigate the usefulness of both indexes. Discusses conditions under which the proposed indexes can be useful. (SLD)

This book is a collection of nine commissioned papers that consider the educational reform represented by performance assessments, their advantages and disadvantages, and the challenges of successful implementation. Both theoretical and practical issues are addressed. (SLD)

The five chapters of this report provide resources that deal with the validity, generalizability, comparability, performance standards, and fairness, equity, and bias of performance assessments. The book is written for experienced educational measurement practitioners, although an extensive familiarity with performance assessment is not required.…

"Measuring Up" describes the current standards-based educational reform movement for a general audience and professionals who have not been involved in reform. The simplified perspective does not do justice to the complexity of reform issues in some contexts, but the case studies illustrating reform efforts are particularly useful. (SLD)

The net effect of the conditions under which tests are taken was empirically investigated using the scores obtained by high school students on an English and a mathematics test. (Author/BB)

A detailed analysis of standard setting and criteria for test scores and educational decisions is presented. The author contends that present procedures are in need of re-examination. (JKS)

Three methods for setting performance standards for criterion referenced tests--theories, expert judgments, and practical necessity--are reviewed. The author concludes that the use of performance standards is not a promising vehicle for making societal decisions. (Author/JKS)

The utility of setting standards for educational decisions, even though those standards may be somewhat arbitrary, is defended in this response to Glass's article (TM 504 031). (JKS)