The turn to online learning and training programs as a response to challenging times (i.e., the COVID-19 crisis) necessitated the need for internet-based testing solutions. Researchers generally have found that Unproctored Internet Testing (UIT) for high-stakes cognitive ability assessments results in higher scores than proctored assessments. Live…

The COVID-19 pandemic severely disrupted assessment models that were commonplace in the testing industry for decades. As a response to this disturbance, remote proctoring has emerged as a promising and potentially sound alternative to offer examinations, while adhering to public health authority guidelines. However, validity evidence in support of…

Since the onset of the pandemic in 2020, many credentialing organizations have incorporated online remote administration of their examinations to enable continuity of their programs. This paper describes a research study examining several high stakes credentialing examination programs that utilized mixed delivery modes, including online remote…

The COVID-19 pandemic has witnessed a renewed interest in Live Remote Proctoring (LRP), not only as a test availability measure but as a necessity to maintain business continuity. Many certification organizations have correspondingly provided LRP as an option for candidates. This study describes a retrospective, observational pilot study…

The rise of online proctoring potentially provides more opportunities for item harvesting and consequent brain dumping and shared "study guides" based on stolen content. This has increased the need for rapid approaches for evaluating and acting on suspicious test responses in every delivery modality. Both hiring proxy test takers and…

Technology-Enhanced Items (TEIs) have been purported to be more motivating and engaging to test takers than traditional multiple-choice items. The claim of enhanced engagement, however, has thus far received limited research attention. This study examined the rates of rapid-guessing behavior received by three types of items (multiple-choice,…

As assessments move from traditional paper-pencil administration to computer-based administration, many testing programs are incorporating alternative item types (AITs) into assessments with the goals of measuring higher-order thinking, offering insight into problem-solving, and representing authentic real-world tasks. This paper explores multiple…

The study aims to introduce catIRT tools which facilitates researchers' Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) simulations. catIRT tools provides an interface for mirt and catR packages through the shiny package in R. Through this interface, researchers can apply IRT calibration and CAT simulations although they do not…

In recent years, Automatic Item Generation (AIG) has increasingly shifted from theoretical research to operational implementation, a shift raising some unforeseen practical challenges. Specifically, generating high-quality answer choices presents several challenges such as ensuring that answer choices blend in nicely together for all possible item…

In this paper we describe content focused, practical procedures to establish benchmarked performance standards for educational and other tests. To implement our procedures, we identify one or more external benchmark (e.g., NAEP or PISA performance levels, college and career readiness benchmarks); link to the external benchmarks to locate those…

With the shift to next generation digital assessments, increased attention has focused on Technology-Enhanced Assessments and Items (TEIs). This study evaluated the feasibility of a high-fidelity digital assessment item response format, which allows students to solve mathematics questions on a tablet using a digital pen. This digital ink approach…

Automatic item generation can rapidly generate large volumes of exam items, but this creates challenges for assembly of exams which aim to include syntactically diverse items. First, we demonstrate a diminishing marginal syntactic return for automatic item generation using a saturation detection approach. This analysis can help users of automatic…

The purpose of this study was to examine the use of a misspecified calibration model and its impact on proficiency classification. Monte Carlo simulation methods were employed to compare competing models when the true structure of the data is known (i.e., testlet conditions). The conditions used in the design (e.g., number of items, testlet to…

When developing a digital test, one of the first decisions that need to be made is which type of Computer-Based Test (CBT) to develop. Six different CBT types are considered here: linear tests, automatically generated tests, computerized adaptive tests, adaptive learning environments, educational simulations, and educational games. The selection…

The educational game industry struggles from a lack of standardization around collecting, analyzing and managing student learning data, which is potentially jeopardizing millions of dollars of investment in educational games. Investors, administrators, and educators require data to support quantitative and qualitative learning measurement in…