NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ796728
Record Type: Journal
Publication Date: 2007
Pages: 22
Abstractor: Author
Reference Count: N/A
ISSN: ISSN-1075-2935
Rating Scale Impact on EFL Essay Marking: A Mixed-Method Study
Barkaoui, Khaled
Assessing Writing, v12 n2 p86-107 2007
Educators often have to choose among different types of rating scales to assess second-language (L2) writing performance. There is little research, however, on how different rating scales affect rater performance. This study employed a mixed-method approach to investigate the effects of two different rating scales on EFL essay scores, rating processes, and raters' perceptions. Four EFL teachers in Tunisia rated a set of 24 EFL essays silently and two subsets of four essays while thinking aloud using a holistic scale and then a multiple-trait rating scale. The essay scores were analyzed using G-theory while the think-aloud protocols were coded in terms of Cuming, Kantor, and Power's (Cumming, A., Kantor, R., & Powers, D. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86 (1), 67-96.) rater decision-making scheme. The holistic scale resulted in higher inter-rater agreement. Raters employed similar processes with both rating scales. Raters were the main source of variability in terms of scores and decision-making behavior. These findings have implications for writing assessment practices and for further research. (Contains 6 tables and 4 figures.)
Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Tunisia