NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ973103
Record Type: Journal
Publication Date: 2012-Jul
Pages: 17
Abstractor: As Provided
Reference Count: N/A
ISSN: ISSN-1075-2935
Using Generalizability Theory to Examine the Accuracy and Validity of Large-Scale ESL Writing Assessment
Huang, Jinyan
Assessing Writing, v17 n3 p123-139 Jul 2012
Using generalizability (G-) theory, this study examined the accuracy and validity of the writing scores assigned to secondary school ESL students in the provincial English examinations in Canada. The major research question that guided this study was: Are there any differences between the accuracy and construct validity of the analytic scores assigned to ESL students and to NE students for the provincial English writing examination across three years? A series of G-studies and decision (D-) studies for three years were conducted to examine accuracy and validity issues. Results showed that differences in score accuracy did exist between ESL and NE students when initial (pre-adjudication) scores were used. The observed G-coefficients for ESL students were significantly lower than those for NE students in all three years, indicating that there were less accuracy and increased errors associated with the writing scores assigned to ESL students. Further, there were significantly less convergent validity in one year and less discriminant validity in all three years of the writing scores assigned to ESL students than to NE students. These findings raise a potential question about the presence of bias in the assessment of ESL students' writing if initial scores were used. (Contains 8 tables.)
Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada