This article describes a novel variation of the Body of Work method that uses construct maps to overcome problems of transparency, rater inconsistency, and scores gaps commonly occurring with the Body of Work method. The Body of Work method with construct maps was implemented to set cut-scores for two separate K-12 assessment programs in a large…

The authors explored the effects of drifting common items on vertical scaling within the higher order framework of item parameter drift (IPD). The results showed that if IPD occurred between a pair of test levels, the scaling performance started to deviate from the ideal state, as indicated by bias of scaling. When there were two items drifting…

In traditional studies on value-added indicators of educational effectiveness, students are usually treated as belonging to those schools where they took their final examination. However, in practice, students sometimes attend multiple schools and therefore it is questionable whether this assumption of belonging to the last school they attended…

Whether an ability test delivered on either paper or computer provides the same information is an important question in applied psychometrics. Besides the validity, it is also the fairness of a measure that is at stake if the test medium affects performance. This study provides a comprehensive review of existing equivalence research in the field…

Typical validation studies on standard setting models, most notably the Angoff and modified Angoff models, have ignored construct development, a critical aspect associated with all conceptualizations of measurement processes. Stone compared the Angoff and objective standard setting (OSS) models and found that Angoff failed to define a legitimate…

Under the No Child Left Behind Act, large-scale test score trend analyses are widespread. These analyses often gloss over interesting changes in test score distributions and involve unrealistic assumptions. Further complications arise from analyses of unanchored, censored assessment data, or proportions of students lying within performance levels…

Discrimination has traditionally been parameterized for items but not other empirical factors. Consequently, if person factors affect discrimination they cause misfit. However, by explicitly formulating the relationship between discrimination and the unit of a metric, it is possible to parameterize discrimination for person groups. This article…

In practice, vertical scales have been continually used to measure students' achievement progress across several grade levels and have been considered very challenging psychometric procedures. Recently, such practices have been drawing many criticisms. The major criticisms focus on dimensionality and construct equivalence of the latent trait or…

From a developmental construct validity perspective, this study examines motivation and engagement across elementary school, high school, and university/college, with particular focus on the Motivation and Engagement Scale (comprising adaptive, impeding/maladaptive, and maladaptive factors). Findings demonstrated developmental construct validity…

This article introduces the Measurement Skills Assessment (MeSA), which was designed to evaluate the mastery of measurement in elementary school students. The primary objectives for the MeSA include covering a broad range of measurement concepts, distinguishing between major subtypes of measurement, and constructing a continuum of items varying in…

In recent years, computer-based testing (CBT) has grown in popularity, is increasingly being implemented across the United States, and will likely become the primary mode for delivering tests in the future. Although CBT offers many advantages over traditional paper-and-pencil testing, assessment experts, researchers, practitioners, and users have…

This study conducted a meta-analysis of computer-based and paper-and-pencil administration mode effects on K-12 student mathematics tests. Both initial and final results based on fixed- and random-effects models are presented. The results based on the final selected studies with homogeneous effect sizes show that the administration mode had no…

As large-scale accountability testing becomes more refined, statewide standards are being created so that teachers and students can create learning and assessment targets that are aligned with statewide testing systems. An important hurdle in assisting teachers in their efforts to create standards-aligned classroom assessments is creating feelings…

Elementary students (n = 277, in Grades 5-6) and high school students (n = 615, in Grades 7-12) responded to 26 items of McInerneys Facilitating Conditions Questionnaire (FCQ). Confirmatory factor analyses of the FCQ found seven distinct factors underlying these items. These were perceived value of schooling (Value), affect toward schooling…

There is agreement that fostering K-12 students' critical thinking is a worthwhile endeavor. However, many educators would agree that there are students in their classrooms who are able to think well but often choose not to utilize those skills. Little is known about the critical thinking dispositions of elementary and secondary students. This…