The ever-changing advances in technology require digital literacy skills for success in the workplace. To determine the critical digital literacy skills needed in the workplace today, the development of a reliable, valid instrument occurred using the nine steps of scale development by DeVellis and Thorpe (2021). Based on the SkillRise (2020a)…

While multimodality and multiliteracies has been a concept for 25 years (Kalantzis & Cope, 2023; The New London Group, 1996), research on and application of the concept within text complexity measures has been limited. Attempts to assess multiliteracies and multimodality (Jacobs, 2013; Schmerbeck & Lucht, 2017; Wyatt-Smith & Kimber,…

Metacognitive awareness is essential in regulating second language (L2) listening and has been predominantly assessed by a multidimensional instrument named the Metacognitive Awareness Listening Questionnaire (MALQ). Since previous studies have yielded inconclusive evidence concerning the generalization of MALQ, it is important to examine the…

In the early years, it has become essential to support the acquisition of computational thinking, which is seen as a 21st-century skill and new literacy. A valid and reliable measurement tool is needed to develop and evaluate educational practices related to these skills. "TechCheck" is a validated unplugged assessment of computational…

We discuss an application of Bayesian factor analysis for estimation of the optimal linear combination and associated maximal reliability of a multi-component measuring instrument. The described procedure yields point and credibility interval estimates of this reliability coefficient, which are readily obtained in educational and behavioral…

The authors contend that children benefit from play as a form of intervention and as a means of fostering their cognitive, social, and physical growth. They review several standardized instruments developed over the last fifty years to assess this benefit of play on child development. They identify twenty-one such play measures, the majority of…

The "LFK" index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing the "LFK" index test to three standard tests for funnel plot asymmetry in settings with smaller or larger…

Mind-mindedness (MM) refers to the parent's ability to treat the child as an individual with a mind of his or her own. Studies have found representational and interactional MM to predict child development, but more research is needed on the validity of representational MM in parents of infants. Therefore, we examine the reliability and validity of…

The second version of Kaiser's Measure of Sampling Adequacy (MSA[subscript 2]) has been widely applied to assess the factorability of data in psychological research. The MSA[subscript 2] is developed in the population and little is known about its behavior in finite samples. If estimated MSA[subscript 2]s are biased due to sampling errors,…

This methodological study aimed to assess the validity and reliability of the Turkish version of the Evaluation of the Impact of Hearing Loss in Adults (ERSA) questionnaire for individuals with treated hearing loss. The study involved 200 participants, and both exploratory factor analysis and confirmatory factor analysis were used to examine…

Purpose: This systematic review and multilevel meta-analysis examines the accuracy of sentence repetition (SR) tasks in distinguishing between typically developing (TD) children and children with developmental language disorder (DLD). It explores variation in the way that SR tasks are administered and/or evaluated and examines whether variability…

Traditional estimators of reliability such as coefficients alpha, theta, omega, and rho (maximal reliability) are prone to give radical underestimates of reliability for the tests common when testing educational achievement. These tests are often structured by widely deviating item difficulties. This is a typical pattern where the traditional…

Background: For peer assessment, reliability (i.e., consistency in ratings across peers) and validity (i.e., consistency of peer ratings with instructors or experts) are frequently examined in the research literature to address a central concern of instructors and students. Although the average levels are generally promising, both reliability and…

Background: Objective Structured Clinical Examinations (OSCEs) are a simulation-based assessment tool used extensively in medical education for evaluating clinical competence. OSCEs are widely regarded as more valid, reliable, and valuable compared to traditional assessment measures, and are now emerging within professional psychology training…

Data extraction is a crucial, yet labor-intensive and error-prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to…