This study adopted a multilevel modeling (MLM) approach to examine the contribution of rater and essay factors to variability in ESL essay holistic scores. Previous research aiming to explain variability in essay holistic scores has focused on either rater or essay factors. The few studies that have examined the contribution of more than one…

Analysis of children's productions of oral narratives provides a rich description of children's oral language skills. However, measures of narrative organization can be directly affected by both developmental and task-based performance constraints which can make a measure insensitive and inappropriate for a particular population and/or sampling…

This paper assesses the performance of a vocabulary test designed to measure second language productive vocabulary knowledge.The test, Lex30, uses a word association task to elicit vocabulary, and uses word frequency data to measure the vocabulary produced. Here we report firstly on the reliability of the test as measured by a test-retest study, a…

This article presents a review of the Canadian Academic English Language (CAEL) Assessment, a high stakes standardized test of the English language. It is a topic-based test that integrates listening, reading, writing and speaking. The test is designed to describe the level of English language proficiency of test takers planning to study at…

Video is widely used in the teaching of L2 listening, and SLA researchers have argued that the visual components of spoken texts are useful for the listener in comprehending aural information. Yet video texts are rarely used on tests of L2 listening ability, perhaps in part due to the belief that including the visual channel involves assessing…

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

This study piloted test items that will be used in a computer-delivered and scored test of productive grammatical ability in English as a second language (ESL). Findings from research on learners' development of morphosyntactic, syntactic, and functional knowledge were synthesized to create a framework of grammatical features. We outline the…

Since the late 1970s, following the major economic reforms that opened the country to the outside world, China has witnessed a growing interest in foreign language education and, hence, a substantial surge in the number of foreign language learners and practitioners. However, the situation of the professional preparation of the practitioners has…

The study reported here examined the evaluation criteria used to assess the proficiency and effectiveness of the language produced in an oral performance test of English conducted in an American university context. Empirical methods were used to analyze qualitatively and quantitatively transcriptions of the Oral English Tests (OET) of 44…

This study investigated the validity of four theoretically motivated traits of writing ability across English and Korean, based on elementary school students' responses to letter- and story-writing tasks. Their responses were scored analytically and analyzed using confirmatory factor analysis. The findings include the following. A model of writing…

Providers of tests of languages for academic purposes generally claim to provide evidence on the extent to which students are likely to be able to cope with the future demands of reading in specified real-life contexts. Such claims need to be supported by evidence that the texts employed in the test reflect salient features of the texts the test…

Previous test fairness frameworks have greatly expanded the scope of fairness, but do not provide a means to fully integrate fairness investigations and set priorities. This article proposes an approach to guide practitioners on fairness research and practices. This approach treats fairness as an aspect of validity and conceptualizes it as…

Despite the growing interest of the language testing community in standard setting, primarily due to the use of the Common European Framework of Reference (CEFR-Council of Europe, 2001), the participants' decision-making process in the CEFR standard setting context remains unexplored. This study attempts to fill in this gap by analyzing these…

This paper presents the author's response to Xiaoming Xi's article titled "How do we go about investigating test fairness?" In this response, the author focuses on test fairness and Toulmin's model of argument structure, Xi's proposal, and the challenges the proposal brings. Xi proposes an approach to investigating test fairness to guide…

This article presents the author's response to Xiaoming Xi's paper titled "How do we go about investigating test fairness?" In the paper, Xi offers "a means to fully integrate fairness investigations and practice". Given the current importance accorded to fairness in the language testing community, Xi makes a case for viewing fairness as an aspect…