This study reviewed conceptualizations and operationalizations of second language (L2) listening constructs. A total of 157 peer-reviewed papers published in 19 journals in applied linguistics were coded for (1) publication year, author, source title, location, language, and reliability and (2) listening subskills, cognitive processes, attributes,…

For institutions where English is the primary language of instruction, English assessments for admissions such as the Test of English as a Foreign Language (TOEFL) and International English Language Testing System (IELTS) give admissions decision-makers a sense of a student's skills in academic English. Despite this explicit purpose, these exams…

Rating scale development in the field of language assessment is often considered in dichotomous ways: It is assumed to be guided either by expert intuition or by drawing on performance data. Even though quite a few authors have argued that rating scale development is rarely so easily classifiable, this dyadic view has dominated language testing…

Many Chinese universities are implementing an educational reform to transform their general English courses into academic English courses. Accordingly, how to assess students' English ability should also be reformed. In this test review, we introduce a school-based general academic English summative test developed by English instructors. An…

Although fairness in assessment practices has gained noticeable attention over the recent years, there has been a long-lasting study to design and validate a questionnaire to measure it from a psychometric perspective. Thus, this study aims to develop and validate a questionnaire with adequate psychometric properties to measure fairness in…

This study aimed at evaluating a PISA-like reading test developed by teachers participating in the teacher training for teaching PISA-like reading. To serve this purpose, an experimental test was administered to 107 students aged 15-16 using a set of text and questions constructed according to the criteria of the PISA Reading test Level 1. Item…

Diagnostic classification models (DCMs) have recently become very popular both for research purposes and for real testing endeavors for student assessment. A plethora of DCM models give researchers and practitioners a wide range of options for student diagnosis and classification. One intriguing option that some DCM models offer is the possibility…

Structural equation modeling (SEM), as a flexible and versatile multivariate statistical technique, has been growingly used since its introduction in the 1970s. This article presents a methodological synthesis of the characteristics of the use of SEM in L2 research by examining the reporting practices in light of the current SEM literature to…

As more various types of computer-assisted language learning (CALL) programs have been incorporated into language classrooms over the recent decades, it has become more important to uncover whether, to what extent, and under which moderator variables CALL can be yield more effective outcomes than traditional language instruction. The issue of…

Argument-based validation requires test developers and researchers to specify what is entailed in test interpretation and use. Doing so has been shown to yield advantages (Chapelle, Enright, & Jamieson, 2010), but it also requires an analysis of how the concerns of language testers can be conceptualized in the terms used to construct a…

The use of assessments that require rater judgment (i.e., rater-mediated assessments) has become increasingly popular in high-stakes language assessments worldwide. Using a systematic literature review, the purpose of this study is to identify and explore the dominant methods for evaluating rating quality within the context of research on…

Drawing on critical discourse analysis methodologies, this study examines the discursive conflicts and strategies employed by newspaper media surrounding the National English Ability Test (NEAT) in Korea. NEAT was reported to cause severe competition among young learners of English and lacked public support. Discursive conflicts over whether NEAT…

Purpose and background: The purpose of this paper is to critically review the traditional and contemporary validation frameworks--the content, criterion, and construct validations; the evidence-gathering; the socio-cognitive model; the test usefulness; and an argument-based approach--as well as empirical studies using an argument-based approach to…

The purpose of this study is to analyze newspaper articles from three publications ("Chosun Ilbo," "Donga Ilbo," and "JoongAng Ilbo") regarding a newly developed high-stakes English test in South Korea, the National English Ability Test (NEAT), from the viewpoint of critical discourse analysis. All of the articles…

Elicited imitation (EI) has been widely used to examine second language (L2) proficiency and development and was an especially popular method in the 1970s and early 1980s. However, as the field embraced more communicative approaches to both instruction and assessment, the use of EI diminished, and the construct-related validity of EI scores as a…