The COVID-19 pandemic has witnessed a renewed interest in Live Remote Proctoring (LRP), not only as a test availability measure but as a necessity to maintain business continuity. Many certification organizations have correspondingly provided LRP as an option for candidates. This study describes a retrospective, observational pilot study…

The turn to online learning and training programs as a response to challenging times (i.e., the COVID-19 crisis) necessitated the need for internet-based testing solutions. Researchers generally have found that Unproctored Internet Testing (UIT) for high-stakes cognitive ability assessments results in higher scores than proctored assessments. Live…

The COVID-19 pandemic severely disrupted assessment models that were commonplace in the testing industry for decades. As a response to this disturbance, remote proctoring has emerged as a promising and potentially sound alternative to offer examinations, while adhering to public health authority guidelines. However, validity evidence in support of…

Since the onset of the pandemic in 2020, many credentialing organizations have incorporated online remote administration of their examinations to enable continuity of their programs. This paper describes a research study examining several high stakes credentialing examination programs that utilized mixed delivery modes, including online remote…

The rise of online proctoring potentially provides more opportunities for item harvesting and consequent brain dumping and shared "study guides" based on stolen content. This has increased the need for rapid approaches for evaluating and acting on suspicious test responses in every delivery modality. Both hiring proxy test takers and…

Feedback is a crucial component of student learning. As advancements in technology have enabled the adoption of digital learning environments with assessment capabilities, the frequency, delivery format, and timeliness of feedback derived from educational assessments have also changed progressively. Advanced technologies powered by Artificial…

Automated Essay Scoring (AES) technologies provide innovative solutions to score the written essays with a much shorter time span and at a fraction of the current cost. Traditionally, AES emphasized the importance of capturing the "coherence" of writing because abundant evidence indicated the connection between coherence and the overall…

The interpretations of test scores in secure, high-stakes environments are dependent on several assumptions, one of which is that examinee responses to items are independent and no enemy items are included on the same forms. This paper documents the development and implementation of a C#-based application that uses Natural Language Processing…

Natural Language Processing (NLP) offers methods for understanding and quantifying the similarity between written documents. Within the testing industry these methods have been used for automatic item generation, automated scoring of text and speech, modeling item characteristics, automatic question answering, machine translation, and automated…

Automated Essay Scoring (AES) using neural networks has helped increase the accuracy and efficiency of scoring students' written tasks. Generally, the improved accuracy of neural network approaches has been attributed to the use of modern word embedding techniques. However, which word embedding techniques produce higher accuracy in AES systems…

This study fit a Naïve Bayesian classifier to the words of exam items to predict the Bloom's taxonomy level of the items. We addressed five research questions, showing that reasonably good prediction of Bloom's level was possible, but accuracy varies across levels. In our study, performance for Level 2 was poor (Level 2 items were misclassified…

A model of cognition and a construct, such as a concept map (Wilson, 2009), is critical in designing assessments of that construct. The Knowledge, Skills and Abilities (KSAs) in the construct must be put to use in order to assess what test takers know and can do (National Research Council, 2001). In order to validate a construct map for graphic…

Technology-Enhanced Items (TEIs) have been purported to be more motivating and engaging to test takers than traditional multiple-choice items. The claim of enhanced engagement, however, has thus far received limited research attention. This study examined the rates of rapid-guessing behavior received by three types of items (multiple-choice,…

As assessments move from traditional paper-pencil administration to computer-based administration, many testing programs are incorporating alternative item types (AITs) into assessments with the goals of measuring higher-order thinking, offering insight into problem-solving, and representing authentic real-world tasks. This paper explores multiple…

The study aims to introduce catIRT tools which facilitates researchers' Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) simulations. catIRT tools provides an interface for mirt and catR packages through the shiny package in R. Through this interface, researchers can apply IRT calibration and CAT simulations although they do not…