Multiple popular math curriculum-based measures have recently been revised in ways that may improve their utility for universal screening. However, the applied use of these tools has yet to be evaluated independently. We conducted a retrospective analysis of the diagnostic accuracy of prior year statewide test results and aimswebPlus math…

The study reported on here contributes to the growing body of knowledge on the use of standard setting methods for improving the reporting and utility value of assessment results in South Africa as well as for addressing the conceptual shortcomings of the Curriculum and Assessment Policy Statement (CAPS) reporting framework. Using data from the…

Research on high-ability postsecondary students has increased in recent years; yet identifying such students can be challenging. The International Cognitive Ability Resource (ICAR) is an online, open-access tool designed to facilitate measurement of cognitive abilities in research. We evaluated whether the ICAR is appropriate to identify…

Rasch and classification analyses on a field-test version of the third edition of the Assessment, Evaluation, and Programming System (AEPS-3), a curriculum-based assessment used to assess young children birth to age 6 years, were conducted. First, an evaluation of the psychometric properties of data from each developmental area of an AEPS-3…

Background/Context: The landscape of college remediation programs experienced significant shifts from prerequisite models to corequisite models in the past few years across the nation. The traditional prerequisite models required students placed below college level to pass a sequence of remedial courses before they could enroll in college-level…

The purpose of this study is to introduce a method for converting scored 4-option multiple-choice (MC) items into scored 3-option MC items without re-pretesting the 3-option MC items. This study describes a six-step process for achieving this goal. Data from a professional credentialing exam was used in this study and the method was applied to 24…

This study aims to demonstrate the optimal way to determine the cut-off score to be used to interpret the total scores obtained from an achievement test or scale using the Artificial Neural Networks method. To this end, the multiple-choice item responses in the Booklet-11 Mathematics subtest at the 8th grade level in the TIMSS 2015 Turkey sample…

Selective cutoff reporting in primary diagnostic accuracy studies with continuous or ordinal data may result in biased estimates when meta-analyzing studies. Collecting individual participant data (IPD) and estimating accuracy across all relevant cutoffs for all studies can overcome such bias but is labour intensive. We meta-analyzed the…

To estimate the long-term effects of the Reading Recovery® intervention, a regression discontinuity design (RD) was implemented in a randomly selected sample of Reading Recovery schools during each year of the federally-funded i3 Scale-Up external evaluation (2011-2015) and also in one additional cohort during the 2016-17 school year. Long-term…

Universal screening to predict students' risk for reading problems is a foundational component of the Multi-Tiered Systems of Support framework and is required by law in many US states. School or district administrators are tasked with selecting screening assessments that are both technically adequate and feasible given the resources of their…

In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

Maintaining equivalent performance standards across forms is a psychometric challenge exacerbated by small samples. In this study, the accuracy of two equating methods (Rasch anchored calibration and nominal weights mean) and four anchor item selection methods were investigated in the context of very small samples (N = 10). Overall, nominal…

Test anxiety has proliferated in the era of test-based accountability; however, there are limited tools that allow for consistent identification of students at risk. The present study reports on the psychometric evidence and continued development of the Multidimensional Test Anxiety Scale (MTAS). Evidence is presented to support both the…

This resource provides data on state and systemwide developmental education assessment and placement policies. Depending on the unique higher education governance structure in each state, a state may have several policies included in this resource. System-level initiatives, programs and collaborations are excluded from this review. A few states…

Background/Context: Pass/fail standardized exams frequently selectively rescore failing exams and retest failing examinees. This practice distorts the test score distribution and can confuse those who do analysis on these distributions. In 2011, the Wall Street Journal showed large discontinuities in the New York City Regent test score…