NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ810675
Record Type: Journal
Publication Date: 2008
Pages: 29
Abstractor: As Provided
Reference Count: 33
ISSN: ISSN-0265-5322
Rater Bias Patterns in an EFL Writing Assessment
Schaefer, Edward
Language Testing, v25 n4 p465-493 2008
The present study employed multi-faceted Rasch measurement (MFRM) to explore the rater bias patterns of native English-speaker (NES) raters when they rate EFL essays. Forty NES raters rated 40 essays written by female Japanese university students on a single topic adapted from the TOEFL Test of Written English (TWE). The essays were assessed using a six-category rating scale (Content, Organization, Style and Quality of Expression, Language Use, Mechanics, and Fluency). MFRM revealed several recurring bias patterns among rater subgroups. In rater-category bias interactions, if Content and/or Organization were rated severely, then Language Use and/or Mechanics were rated leniently, and vice versa. In rater-writer bias interactions, there tended to be more severe or lenient bias towards higher ability writers than lower ability writers. Some raters also rated higher ability writers more severely and lower ability writers more leniently than expected. This study has implications for issues of rater training in L2 writing assessment. (Contains 4 figures and 6 tables.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Japan