Many innovative item formats have been proposed over the past decade, but little empirical research has been conducted on their measurement properties. This study examines the reliability, efficiency, and construct validity of two innovative item formats--the figural response (FR) and constructed response (CR) formats used in a K-12 computerized…

There have been many studies of the comparability of computer-administered and paper-administered tests. Not surprisingly (given the variety of measurement and statistical sampling issues that can affect any one study) the results of such studies have not always been consistent. Moreover, the quality of computer-based test administration systems…

In cut score setting processes, subject matter experts are asked to make judgments about the likely performance of examinees at a targeted skill level. When cut scores are used in K-12 settings to separate students who have and have not mastered certain skills, the target examinee may be characterized as the barely proficient or barely master…

Extrapolates from existing case law on educational testing to the likely ways in which school accountability systems may be challenged in the courts. Also offers a list of recommended actions for accountability system developers. (SLD)

Reviews and illustrates some of the current technological developments in computer-based testing, focusing on novel item formats and automated scoring methodologies. The review shows a number of innovations being researched and implemented. (SLD)

Calculated year-1 to year-2 stability of assessment data from 21 states and 2 Canadian provinces. The median stability coefficient was 0.78 in mathematics and reading, and lower in writing. A stability coefficient of 0.80 is recommended as the standard for large-scale assessments of student performance. (SLD)

Analyzes five studies on the effects of extra examination time for students with learning disabilities. These studies directly tested the Maximum Potential Thesis, but show only weak support for it. Discusses some methodological problems that weaken these studies. (SLD)

Provides a chronological summary of the evolution of statewide achievement tests in Texas to facilitate understanding of the issues behind the "GI Forum v. Texas Education Agency" litigation (1999), which challenged the curricular links of the Texas Assessment of Academic Skills and other "opportunity to learn" issues. (SLD)

Discusses the implications of the "GI Forum v. Texas Education Agency" (2000) decision supporting the use of the Texas Assessment of Academic Skills for other state assessment programs. Notes that the legal success of the state test in Texas was not due merely to good fortune, but to the test's adherence to legally defensible principles. (SLD)

Describes examinee-centered method for setting multiple cutscores on a test involving both objective and extended-response items. Judges evaluate a representative sample of examinee performance using a rating scale that is defined in terms of performance standards, and these ratings are linked to examinee's test scores to generate a functional…

Examined the stability of the standard error of measurement and the relationship between the reliability coefficient and the variance of both true scores and error scores for 170 school districts in a state. As expected, reliability coefficients varied as a function of group variability, but the variation in split-half coefficients from school to…

Two studies involving 368 elementary and high school students and 29 readers were conducted to investigate reader consistency, score reliability, and reader time requirements of three hands-on science performance tasks. Holistic scores were as reliable as analytic scores, and there was a high correlation between them after they were disattenuated…

The process used to identify the provisional locations of points that would inform the National Assessment Governing Board in setting achievement levels for the science National Assessment of Educational Progress (NAEP) is described. This process includes identification of panelists for achievement-level setting and the method for converting…

The validity of interpretations of National Assessment of Educational Progress (NAEP) achievement levels is evaluated by focusing on evidence regarding three types of discrepancies: (1) between standards; (2) among descriptions of achievement levels; and (3) between assessments and content standards. All of these discrepancies raise serious…

How summarizing National Assessment of Educational Progress (NAEP) results in terms of a market basket of items would affect achievement-level reporting is discussed. A market basket is a specific set of items one may administer, scores on which constitute a reporting scale. Advantages and limitations of the market-basket approach are discussed.…