Possible lack of motivation is a perpetual concern when tests have no stakes attached to performance. Specifically, the validity of test score interpretations may be compromised when examinees are unmotivated to exert their best efforts. Motivation filtering, a procedure that filters out apparently unmotivated examinees, was applied to the…

Many innovative item formats have been proposed over the past decade, but little empirical research has been conducted on their measurement properties. This study examines the reliability, efficiency, and construct validity of two innovative item formats--the figural response (FR) and constructed response (CR) formats used in a K-12 computerized…

The evidence gathered in the present study supports the use of the simultaneous development of test items for different languages. The simultaneous approach used in the present study involved writing an item in one language (e.g., French) and, before moving to the development of a second item, translating the item into the second language (e.g.,…

The National Assessment Governing Board used a new method to set achievement level standards on the 2005 Grade 12 NAEP Math test. In this article, we summarize our independent evaluation of the process used to set these standards. The evaluation data included observations of the standard-setting meeting, observations of advisory committee meetings…

Previous research has shown that rapid-guessing behavior can degrade the validity of test scores from low-stakes proficiency tests. This study examined, using hierarchical generalized linear modeling, examinee and item characteristics for predicting rapid-guessing behavior. Several item characteristics were found significant; items with more text…

Under Title III of the No Child Left Behind (NCLB) Act of 2001 (NCLB, 2001b) every state needs to show linkage between state content standards and state English language development standards as input to the development of state English proficiency tests. This article argues that Title III presents a unique opportunity to explore how different…

A number of vertical scaling methodologies were examined in this article. Scaling variations included data collection design, scaling method, item response theory (IRT) scoring procedure, and proficiency estimation method. Vertical scales were developed for Grade 3 through Grade 8 for 4 content areas and 9 simulated datasets. A total of 11 scaling…

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

This study examined the effect of oral administration accommodations on test structure and student performance on the mathematics portion of the South Carolina High School Exit Examination (HSEE). The examination was given at Grade 10 and was untimed. Three groups of students were studied. Two groups took the regular form. One group had recorded…

This research examined the degree of comparability and sources of incomparability of English and French versions of reading, mathematics, and science tests that were administered as part of a survey of achievement in Canada. The results point to substantial psychometric differences between the 2 language versions. Approximately 18% to 36% of the…

Two models are presented in this article for estimating the proportion of students who would pass all of three or more content area tests given that none have actually been tested in more than two of the content areas. The first model allows one to estimate the proportion of students who would pass all of three or more content area tests from the…

Examined the precision of change scores in terms of error-tolerance (E/T) ratios for both relative and absolute interpretations of change scores. Used E/T ratios to evaluate the error in estimating the change relative to tolerance for error in a particular context. Illustrates the results with achievement test data. (SLD)

Provides a chronological summary of the evolution of statewide achievement tests in Texas to facilitate understanding of the issues behind the "GI Forum v. Texas Education Agency" litigation (1999), which challenged the curricular links of the Texas Assessment of Academic Skills and other "opportunity to learn" issues. (SLD)

Describes the Texas test development process in detail, showing how each test development step is linked to the "Standards for Educational and Psychological Testing." The routine use of this process provides evidence of the content and curricular validity of the Texas Assessment of Academic Skills. (SLD)

Describes approaches taken in Texas to bring about academic accountability for students of limited English proficiency through evaluating and reporting annually on their progress in English-language literacy and their learning of school subjects and by documenting the growth in successful performance on state tests by this special population. (SLD)