NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ948651
Record Type: Journal
Publication Date: 2011-Dec
Pages: 21
Abstractor: As Provided
Reference Count: 37
ISSN: ISSN-1949-3533
Generalizability Theory as Evidence of Concerns about Fairness in Large-Scale ESL Writing Assessments
Huang, Jinyan
TESOL Journal, v2 n4 p423-443 Dec 2011
Using generalizability theory, this study examined both the rating variability and reliability of English as a second language (ESL) students' writing in two provincial examinations in Canada. This article discusses expected and unexpected similarities and differences related to rating variability and reliability between the two testing programs. As expected, there was more desired but less unwanted variation in ESL and native-English-speaking (NES) students' writing scores in Province B than in Province A. But unexpectedly, the results demonstrated systematic differences between ESL and NES students in terms of rating variability. Further, there were lower reliabilities in ESL students' scores in comparison to NES students' scores. These findings raise potential concerns about the fairness of large-scale ESL writing assessments. (Contains 4 tables and 1 footnote.)
Teachers of English to Speakers of Other Languages, Inc. 1925 Ballenger Avenue Suite 550, Alexandria, VA 22314. Tel: 888-547-3369; Tel: 703-836-0774; Fax: 703-836-7864; Fax: 703-836-6447; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 12; High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada