Multiple-choice items are a mainstay of achievement testing. The need to adequately cover the content domain to certify achievement proficiency by producing meaningful precise scores requires many high-quality items. More 3-option items can be administered than 4- or 5-option items per testing time while improving content coverage, without…

A sample of 293 local district assessments used in the Nebraska STARS (School-based Teacher-led Assessment and Reporting System), 147 from 2004 district mathematics assessment portfolios and 146 from 2003 reading assessment portfolios, was scored with a rubric evaluating their quality. Scorers were Nebraska educators with background and training…

An assumption that is fundamental to the scoring of student-constructed responses (e.g., essays) is the ability of raters to focus on the response characteristics of interest rather than on other features. A common example, and the focus of this study, is the ability of raters to score a response based on the content achievement it demonstrates…

The purpose of a credentialing examination is to assure the public that individuals who work in an occupation or profession have met certain standards. To be consistent with this purpose, credentialing examinations must be job related, and this requirement is typically met by developing test plans based on an empirical job or practice analysis.…

This research investigates the relative importance of vocabulary and oral reading fluency as measurement dimensions of reading comprehension as the student passes from elementary to high school. Invariance of this model over grades 4 through 8 is tested using two independent student samples reading grade-level appropriate passages. Results from…

With 3 years and counting since its inception, the scope and impact of "No Child Left Behind" is now being felt in classrooms across the nation. Although some successes have been identified, concerns about the implementation and expectations of the legislation are emerging. As a result of the legislation's emphasis on the development of standards…

The author takes the position that many of the fundamental measurement concepts and relationships often are used in our written communications in ways that demonstrate misunderstanding and lead to confusion. Illustrations of the misuses are provided, anecdotes and speculation about the implications are offered, and suggestions for addressing the…

There are many threats to validity in high-stakes achievement testing. One major threat is construct-irrelevant variance (CIV). This article defines CIV in the context of the contemporary, unitary view of validity and presents logical arguments, hypotheses, and documentation for a variety of CIV sources that commonly threaten interpretations of…

The validity of the SAT as an admissions criterion for Latinos and Asian Americans who are not native English speakers was examined. The analyses, based on 1997 and 1998 UCSB freshmen, focused on the effectiveness of SAT scores and high school grade-point average (HSGPA) in predicting college freshman grade-point average (FGPA). When regression…

Research on psychometric methods is heavily dependent on software. The quality, availability, and documentation of such software are critical to the advancement of the field. In 2000, an ad hoc committee of NCME recommended that NCME adopt policies that promote greater availability and better documentation of software. This article follows the ad…

Nebraska's approach to standards, assessment, and accountability, the School-based Teacher-led Assessment and Reporting System (STARS) is based upon local control and the belief that classrooms and teachers must be at the heart of student learning and accountability. STARS relies on locally-developed assessment systems to accurately measure and…

Nebraska districts use different strategies for measuring student performance on the state's content standards. District assessments differ in type and technical quality. Six quality criteria were endorsed by the state. These criteria cover content and curricular validity, fairness, and appropriateness of score interpretations. District assessment…

As part of Nebraska's assessment and accountability system, districts' local assessment systems are evaluated for their psychometric quality. This article provides an overview of a two-stage evaluation strategy, discusses how it was applied in Nebraska, and presents results from the first three years of the evaluation process. Benefits of the…

When STARS reform efforts were launched in 2000, teacher training in assessment was seen as crucial to the success of the program. The STARS reform efforts focus on both supporting the implementation of quality classroom assessment practices and implementing a district-based accountability system. The training programs described in this article…

Recent implementation of Nebraska's Standards-based Teacher-led Assessment and Reporting System (STARS) introduced a unique opportunity to examine the benefits and drawbacks of a teacher-led state assessment system. STARS is unique among state assessment systems in that statewide tests are replaced by locally developed assessments designed by…