NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1055846
Record Type: Journal
Publication Date: 2015
Pages: 5
Abstractor: ERIC
ISSN: ISSN-1536-6367
Validity Inferences under High-Stakes Conditions: A Response from Language Testing
Hill, Kathryn; McNamara, Tim
Measurement: Interdisciplinary Research and Perspectives, v13 n1 p39-43 2015
Those who work in second- and foreign-language testing often find Koretz's concern for validity inferences under high-stakes (VIHS) conditions both welcome and familiar. While the focus of the article is more narrowly on the potential for two instructional responses to test-based accountability, "reallocation" and "coaching," to undermine the validity of score-based inferences about achievement by inflating scores and exaggerating mastery of the domain of interest, rather than the broader set of problems around test-based accountability (Haertel, 2013), the issues raised have also been extensively discussed for several years in the field of language testing, using somewhat different terminology. "Washback" (or "backwash") is the specific term used to refer to the effects of high-stakes assessment on teaching and learning and is considered a subset of test impact more generally (Cheng, 2008). The inflationary or deflationary effect of nonsubstantive (construct irrelevant) "performance elements" on scores (associated with coaching in the article) is known as "test method effect" (Bachman, 1990). The context of the article, where outcomes-based control of educational effort is found to have a distorting effect on teaching and learning efforts, has many parallels in language testing, not only in the United States, where the negative impact on bilingual education of test-based accountability under No Child Left Behind and more recently the Common Core Standards has been widely discussed (Menken, 2008), but internationally, particularly in studies of the problematic impact of the Common European Framework of Reference (Council of Europe 2001) and PISA testing on language education (McNamara, 2011). Issues around validity inferences under high-stakes conditions arise in language testing in more contexts than the kind of test based accountability scenarios that are the focus of this article. Since the appearance of a special volume in "Language Testing" (the leading journal in the field) featuring an article on washback by Messick (1996), there has been a significant amount of research undertaken on washback and the broader impacts of language assessment, including a suite of studies sponsored by the major language testing agencies themselves. However, the high-stakes language assessments forming the focus of these studies are typically used for selection and/or accreditation purposes rather than for school and teacher accountability: language tests act as gatekeeping devices to control access to employment, education, migration, and citizenship. There is a general consensus in language-testing research that high-stakes language assessment influences "what" is taught but not necessarily "how" it is taught.
Psychology Press. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Opinion Papers; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Laws, Policies, & Programs: No Child Left Behind Act 2001
Identifiers - Assessments and Surveys: Program for International Student Assessment; Test of English as a Foreign Language
Grant or Contract Numbers: N/A