ERIC Number: EJ796728
Record Type: Journal
Publication Date: 2007
Pages: 22
Abstractor: Author
ISBN: N/A
ISSN: ISSN-1075-2935
EISSN: N/A
Rating Scale Impact on EFL Essay Marking: A Mixed-Method Study
Barkaoui, Khaled
Assessing Writing, v12 n2 p86-107 2007
Educators often have to choose among different types of rating scales to assess second-language (L2) writing performance. There is little research, however, on how different rating scales affect rater performance. This study employed a mixed-method approach to investigate the effects of two different rating scales on EFL essay scores, rating processes, and raters' perceptions. Four EFL teachers in Tunisia rated a set of 24 EFL essays silently and two subsets of four essays while thinking aloud using a holistic scale and then a multiple-trait rating scale. The essay scores were analyzed using G-theory while the think-aloud protocols were coded in terms of Cuming, Kantor, and Power's (Cumming, A., Kantor, R., & Powers, D. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86 (1), 67-96.) rater decision-making scheme. The holistic scale resulted in higher inter-rater agreement. Raters employed similar processes with both rating scales. Raters were the main source of variability in terms of scores and decision-making behavior. These findings have implications for writing assessment practices and for further research. (Contains 6 tables and 4 figures.)
Descriptors: Writing Evaluation, Writing Tests, Rating Scales, Essays, Foreign Countries, Teaching Methods, English (Second Language), Second Language Learning, Scores, Protocol Analysis, Interrater Reliability, Decision Making
Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Tunisia
Grant or Contract Numbers: N/A