In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers' ratings of their own candidates' performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of…

The study of measurement invariance in polytomous items that targets individual score levels is known as differential step functioning (DSF). The analysis of DSF requires the creation of a set of dichotomizations of the item response variable. There are two primary approaches for creating the set of dichotomizations to conduct a DSF analysis: the…

Item-level information, such as difficulty and discrimination are invaluable to the test assembly, equating, and scoring practices. Estimating these parameters within the context of large-scale performance assessments is often hindered by the use of unbalanced designs for assigning examinees to tasks and raters because such designs result in very…

The current study examines how using a multidimensional battery of predictors (high-school grade point average (GPA), SAT/ACT, and biodata), and weighting the predictors based on the different values institutions place on various student performance dimensions (college GPA, organizational citizenship behaviors (OCBs), and behaviorally anchored…

Standardized assessment in school systems has been the center of debate for decades. Although the voices of opponents of standardized tests have dominated the public forum, only a handful of scholars and practitioners have argued in defense of standardized tests. This article provides a critical synthesis of the controversial issues on…

Two concerns related to setting performance standards on educational assessments are discussed. First, criteria for a standard-setting process from the point of view of a standard-setting sponsor, called here "institutional criteria," are developed using a state department of education as an example. Four institutional criteria are proposed: (a)…

Attempts to place the issues associated with computer-automated scoring within the context of current validity theory and presents a taxonomy of automated scoring procedures as a framework for discussing threats to validity that may take on increased importance for specific approaches to automated scoring. (SLD)

Attempted to improve the measurement quality of a complex performance assessment through principled assessment design using the example of the National Board for Professional Teaching Standards Early Childhood/Generalist examination. All indexes examined improved after revisions were made. Results show the importance of attention to assessment…

To determine the factors that contribute to or compromise the effectiveness of multiscored items, this study combined analysis of statewide score data from the 1996 Maryland School Performance Assessment Program tests with systematic analyses of 60 activities providing measures of writing, language usage, or both, and one or more content areas.…

Studied the sampling variability of estimated variance components using data collected over several years for a listening and writing performance assessment and evaluated the stability of estimated measurement precision. Results indicate that the estimated variance components varied from one year to another and suggest that the measurement…

Examined the effects of providing students with background information about the structure and scoring of mathematics performance assessments (PA). Results for 187 elementary school students who had PA orientation and 182 who did not show the effects of test wiseness training for average and above-average students, but not for below-average…

Studied content domain, format, and level of inquiry as factors contributing to the large variation in student performance across open-ended measures. Results for more than 1,200 eighth graders do not support the hypothesis that tasks similar in content, format, and level of inquiry would correlate higher with each other than with measures…

Two studies involving 368 elementary and high school students and 29 readers were conducted to investigate reader consistency, score reliability, and reader time requirements of three hands-on science performance tasks. Holistic scores were as reliable as analytic scores, and there was a high correlation between them after they were disattenuated…

The temporal stability and intertask consistency of an eighth-grade mathematics performance assessment and how task similarity affects the ability to generalize results of the assessments were studied with results from 101 eighth graders. Results support the suggestion that large-scale performance assessments be used with considerable caution…

Credentialing programs were surveyed to determine the procedures they use to set performance standards on multiple-choice and open-ended assessments. Implications of the various standard-setting approaches for the National Assessment of Educational Progress are discussed, and it is asserted that generalizing from standard-setting in professional…