The template-based automated item-generation (TAIG) approach that involves template creation, item generation, item selection, field-testing, and evaluation has more steps than the traditional item development method. Consequentially, there is more margin for error in this process, and any template errors can be cascaded to the generated items.…

When designing or modifying a test, an important challenge is controlling its speededness. To achieve this, van der Linden (2011a, 2011b) proposed using a lognormal response time model, more specifically the two-parameter lognormal model, and automated test assembly (ATA) via mixed integer linear programming. However, this approach has a severe…

A recurring question regarding Likert items is whether the discrete steps that this response format allows represent constant increments along the underlying continuum. This question appears unsolvable because Likert responses carry no direct information to this effect. Yet, any item administered in Likert format can identically be administered…

Multidimensional achievement tests are recently gaining more importance in educational and psychological measurements. For example, multidimensional diagnostic tests can help students to determine which particular domain of knowledge they need to improve for better performance. To estimate the characteristics of candidate items (calibration) for…

Short scales are time-efficient for participants and cost-effective in research. However, researchers often mistakenly expect short scales to have the same reliability as long ones without considering the effect of scale length. We argue that applying a universal benchmark for alpha is problematic as the impact of low-quality items is greater on…

Combining methods from earlier content validity and more contemporary content alignment studies may allow a more complete evaluation of the meaning of test scores than if either set of methods is used on its own. This article distinguishes item relevance indices in the content validity literature from test representativeness indices in the…

The purpose of this study was originally to create an operational definition of the "appearance of competence" to design valid questions for educational professionals supporting twice-exceptional (2e) learners to create a testing instrument. Through the methodological process of grounded theory, a replacement research question emerged as…

Question generation (QG) task plays a crucial role in adaptive learning. While significant QG performance advancements are reported, the existing QG studies are still far from practical usage. One point that needs strengthening is to consider the generation of question group, which remains untouched. For forming a question group, intrafactors…

Questionnaire designers are encouraged to write questions as complete sentences. In self-administered surveys, incomplete question stems may reduce visual clutter but may also increase burden when respondents need to scan the response options to fully complete the question. We experimentally examine the effects of three categories of incomplete…

Purpose: This paper analyses and classifies currently available English- and German-language measurement instruments for assessing economic literacy. It shows the content-related focuses and gaps of the extracted test instruments, the cognitive level of demand that characterises the instruments, the technical forms of implementation, and the…

Empowerment is a concept that has become increasingly used over recent years. However, little research has been undertaken into how empowerment can be evaluated, particularly in the case of young people. The aim of this article is to present an inventory of dimensions and indicators of youth empowerment. The article describes the various phases in…

Conceptualizing and testing factors that contribute to the success of community-academic partnerships are critical to understanding their contributions to the health and well-being of communities. Most measures to date focus on factors that contribute to the development of new partnerships, and only a few have been adequately tested and validated.…

Stratification is an important design feature of many studies using complex sampling designs and it is often used in large-scale assessment (LSA) studies, such as the "Programme for International Student Assessment" (PISA), for two main reasons. First, stratification variables that achieve a high between and low within strata variance…

Evaluation data is needed to demonstrate the impact of 4-H science, technology, engineering, and mathematics (STEM) programming on children and youth. However, collecting evaluation data from cloverbuds (ages 5-7) is particularly challenging given their developmental age. We adapted an observational Cloverbud evaluation tool to measure basic life…

Teachers play a critical role in school mental health. They aid in the identification and referral of students in need of mental health services and are key players in implementing interventions. Nevertheless, teachers often lack the education and training needed to support youths' mental health. Increasing teachers' mental health literacy (MHL)…