In this article, we consolidate and present in one place what is known about quality indicators for setting standards so that stakeholders may be able to recognize the signs of standard-setting quality. We use the context of setting standards to associate English language test scores with language proficiency descriptions such as those presented…

This study explores the construct of a summarization test task by means of single-group and multigroup structural equation modeling (SEM). It examines the interrelationships between strategy use and performance, drawing on data from 298 Taiwanese undergraduates' summary essays and their self-reported strategy use. Single-group SEM analyses…

The Yes/No Angoff and Bookmark method for setting standards on educational assessment are currently two of the most popular standard-setting methods. However, there is no research into the comparability of these two methods in the context of language assessment. This study compared results from the Yes/No Angoff and Bookmark methods as applied to…

This study, which is part of a large-scale study of using objective measures to validate assessment rating scales and assessment tasks in a high-profile school-based assessment initiative in Hong Kong, examined how grammatical complexity measures relate to task type and analytic evaluations of students' speaking proficiency in a classroom-based…

This article proposes a distinction between "fairness" and "justice" in relation to language tests. Basing its discussion on the validity theory of Samuel Messick, it discusses the way in which these terms have been used in the existing literature, and their relationship to the concept of validity. Fairness, broadly speaking, refers to the…

Professor Elana Shohamy was born in New York and moved to Israel at a young age. It was an interesting time for the country and the Hebrew language. In this article, Professor Shohamy shares her keen insights into her experiences growing up in Israel. She obtained her teacher's certificate from David Yelin Teacher's College in Jerusalem,…

This study examines score variations and differences in the reliability of ratings between English-as-a-second-language (ESL) and native English (NE) authored papers in a graduate course. Generalizability (G-) theory was used as a framework for analysis because it is powerful in detecting rater variability and the relative contributions of…

The study described in this article investigated the relationship between three variables in the IELTS oral module--planning, proficiency, and task--and was designed to enhance our understanding of how or whether these variables interact. The study aimed to determine whether differences in performance resulted from 1 or 2 min of planning time. It…

In researching the potential of cognitive diagnostic assessment, researchers concur that the quality of diagnostic inferences is subject to the extent to which the construct representations based on cognitive skills are theoretically compelling, empirically sound, and relevant to test use. In this paper, I argue that the construction of a Q matrix…

I welcome the feedback from Downey et al. on my analysis of the "Versant for English" Test (formerly called the PhonePass Test), as it provides opportunities to engage in dialogue on differing notions of language performance and authenticity and address the implications of technological expediency influencing and shaping the design of test tasks.…

The increasing popularity of paired format in oral testing has engendered legitimate scrutiny of its reliability and validity as compared with the more traditional interviewer-interviewee arrangement. Although characteristics such as the gender, cultural/L1 background, and language proficiency of one's interlocutor likely affects the discourse…

When testing English language learners (ELLs) in subject matter areas, construct irrelevant variance could result from English, the language in which the test is presented. Differential item functioning (DIF) techniques have been used to determine if items are operating differently for population subgroups and might therefore be appropriate for…

From June to November 2003 an exploratory survey was made of the assessment of written English in a small sample of English language degrees in Europe. All programmes had similar components, the study of English language and literature, cultural or area studies, and English language development. Some involved the study of another language.…

The Multimedia Assisted Test of English Speaking was designed to assess global speaking competence of Korean speakers of English at Sookmyung Women's University. The test was developed with the help of the American Council on the Teaching of Foreign Languages for their proficiency guidelines and of the Center for Applied Linguistics for their…

The thought of Samuel Messick has influenced language testing in 2 main ways: in proposing a new understanding of how inferences made based on tests must be challenged, and in drawing attention to the consequences of test use. The former has had a powerful impact on language-testing research, most notably in Bachman's work on validity and the…