The purpose of this instructional piece was to provide a nontechnical synthesis of common internal consistency reliability estimates used in professional counseling and in related fields. The article begins with an overview of coefficients alpha, omega, omega hierarchical, and H, with guidelines for their selection. Next, I provide recommendations…

Cognitive diagnostic models (CDMs) provide pedagogically relevant information in the form of a student profile of multiple binary categorizations of students into mastery or nonmastery statuses on latent traits called attributes. Federal educational accountability requires accountability measures to designate students into one of at least three…

Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

The use of brief dyslexia rating scales is increasing given current dyslexia legislation efforts across the United States. The purpose of this article is to provide an overview of the historical context of the use of brief dyslexia rating scales, strengths, and limitations of using these measures, criteria for selecting these measures, and a…

Given the recent push for universal screening, it is important to take into account how well a screener identifies children at risk for reading problems as well as how screener and sample information contribute to this classification. Picking the best cut-point for a particular sample and screening goal can be challenging given that test manuals…

Given the recent push for universal screening, it is important to take into account how well a screener identifies children at risk for reading problems as well as how screener and sample information contribute to this classification. Picking the best cut-point for a particular sample and screening goal can be challenging given that test manuals…

A recent review found that 11% of published factor models are hierarchical models with second-order factors. However, dedicated recommendations for evaluating hierarchical model fit have yet to emerge. Traditional benchmarks like RMSEA <0.06 or CFI >0.95 are often consulted, but they were never intended to generalize to hierarchical models.…

Subjects with initially extreme observations upon remeasurement are found closer to the population mean. This tendency of observations toward the mean is called regression to the mean (RTM) and can make natural variation in repeated data look like real change. Studies, where subjects are selected on a baseline criterion, should be guarded against…

Commonly used cut-points may misclassify physical activity (PA) in people with cystic fibrosis (CF). The aim of this study was to develop and cross-validate condition-specific cut-points in children and adolescents with CF. Thirty-five children and adolescents with CF (15 girls; 11.6 ± 2.8 years) and 28 controls (16 girls; 12.2 ± 2.7 years), had…

The most common effect size when using a multiple-group confirmatory factor analysis approach to measurement invariance is [delta]CFI and [delta]TLI with a cutoff value of 0.01. However, this recommended cutoff value may not be ubiquitously appropriate and may be of limited application for some tests (e.g., measures using dichotomous items or…

Setting cutoff scores is one of the most common practices when using scales to aid in classification purposes. This process is usually done univariately where each optimal cutoff value is decided sequentially, subscale by subscale. While it is widely known that this process necessarily reduces the probability of "passing" such a test,…

Appropriate placement into courses at postsecondary institutions is critical for the success of students in terms of retention and graduation rates. To reduce the number of students who are misplaced, using multiple measures in placing students is encouraged. However, in practice most postsecondary schools utilize only a few measures to determine…

Purpose: The purpose of this research is to address the need for a robust system to accurately determine a cutoff score by using the Angoff method and leveraging the Rasch infit and outfit statistics of item response theory by detecting and removing misfitting items in a test. Design/methodology/approach: Researchers in educational evaluation…

The subjective aspect of standard-setting is often criticized, yet data-driven standard-setting methods are rarely applied. Therefore, we applied a mixture Rasch model approach to setting performance standards across several testing programs of various sizes and compared the results to existing passing standards derived from traditional…

Purpose: This study examined the discriminative ability of acoustic indices of vocal hyperfunction combining smoothed cepstral peak prominence (CPPS) and relative fundamental frequency (RFF). Method: Demographic, CPPS, and RFF parameters were entered into logistic regression models trained on two 1:1 case-control groups: individuals with and…