Regular expression (regex) coding has advantages for text analysis. Humans are often able to quickly construct intelligible coding rules with high precision. That is, researchers can identify words and word patterns that correctly classify examples of a particular concept. And, it is often easy to identify false positives and improve the regex…

This study examined the predictive validity and classification accuracy of individual- and group-administered screening measures relative to student performance on a year-end state reading assessment in two states. A sample of 321 students was assessed in the areas of word-level and text fluency, as well as reading comprehension in the fall of…

Assessment instruments composed of two-tier multiple choice (TTMC) items are widely used in science education as an effective method to evaluate students' sophisticated understanding. In practice, however, there are often concerns regarding the common scoring methods of TTMC items, which include pair scoring and individual scoring schemes. The…

Performance assessments, scenario-based tasks, and other groups of items carry a risk of violating the local item independence assumption made by unidimensional item response theory (IRT) models. Previous studies have identified negative impacts of ignoring such violations, most notably inflated reliability estimates. Still, the influence of this…

This study examined the predictive validity and classification accuracy of individual and group-administered screening measures relative to student performance on a year-end state reading assessment in two states. A sample of 321 students were assessed in the areas of word-level and text fluency, as well as reading comprehension in the fall of…

Business analytics courses, such as marketing research, data mining, forecasting, and advanced financial modeling, have substantial predictive modeling components. The predictive modeling in these courses requires students to estimate and test many linear regressions. As a result, false positive variable selection ("type I errors") is…

Contract cheating happens when students outsource their assessed work to a third party. One approach that has been suggested for improving contract cheating detection is comparing students' assignment submissions with their previous work, the rationale being that changes in style may indicate a piece of work has been written by somebody else. This…

In 2016, Uruguay started gathering longitudinal student data to improve educational trajectories by putting in place an 'early alert' system. Underlying the system is the understanding that prior schooling predicts likelihood of grade repetition and grade repetition predicts later school dropout, while close follow-up can help prevent both…

Erasure analysis is the study of the pattern or quantity of erasures on multiple-choice paper-and-pencil examinations, to determine whether erasures were made post-testing for the purpose of unfairly increasing students' scores. This study examined the erasure data from over 1.4 million exams, taken by more than 600,000 students. Three…

Automated computerized scoring systems (ACSSs) are being increasingly used to analyze text in many educational settings. Nevertheless, the impact of misspelled words (MSW) on scoring accuracy remains to be investigated in many domains, particularly jargon-rich disciplines such as the life sciences. Empirical studies confirm that MSW are a…

In psychological and survey research, the prevalence and serious consequences of careless responses from unmotivated participants are well known. In this study, we propose to iteratively detect careless responders and cleanse the data by removing their responses. The careless responders are detected using person-fit statistics. In two simulation…

The authors assessed the extent to which multielement designs produced false positives using continuous duration recording (CDR) and interval recording with 10-s and 1-min interval sizes. Specifically, they created 6,000 graphs with multielement designs that varied in the number of data paths, and the number of data points per data path, using a…

Many studies have shown that using a computer-aided detection (CAD) system does not significantly improve diagnostic accuracy in radiology, possibly because radiologists fail to interpret the CAD results properly. We tested this possibility using screening mammography as an illustrative example. We carried out two experiments, one using 28…

This article describes the development of a tense and aspect identifier, an online tool designed to help learners of English by harnessing a natural language processing pipeline to automatically classify verb groups into one of 12 grammatical tenses. Currently, there is no website or application that can automatically identify tense in context,…

This article illustrates the value of large-scale impact evaluations with counterfactual components. It begins by exploring the limitations of small-scale impact studies, which do not allow reliable inference to a wider population or which do not use valid comparison groups. The paper then describes the design features of a recent large-scale…