NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ934580
Record Type: Journal
Publication Date: 2011
Pages: 15
Abstractor: As Provided
Reference Count: 32
ISSN: ISSN-0969-594X
Effects of Marking Method and Rater Experience on ESL Essay Scores and Rater Performance
Barkaoui, Khaled
Assessment in Education: Principles, Policy & Practice, v18 n3 p279-293 2011
This study examined the effects of marking method and rater experience on ESL (English as a Second Language) essay test scores and rater performance. Each of 31 novice and 29 experienced raters rated a sample of ESL essays both holistically and analytically. Essay scores were analysed using a multi-faceted Rasch model to compare test-takers' writing ability estimates, inter-rater agreement, and rater severity and self-consistency across marking methods (holistic vs. analytic) and rater groups (novice vs. experienced). Raters tended to be less severe with analytic marking. Holistic marking led to a higher level of inter-rater agreement, while analytic marking led to higher self-consistency. The two rater groups differed in terms of severity, inter-rater agreement and self-consistency. Furthermore, the two marking methods performed differently across the two rater groups. Overall, the findings suggest that the two marking methods might be useful for different assessment purposes, contexts and raters. Implications for ESL writing assessment practice and research are discussed. (Contains 5 notes and 3 tables.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Adult Education; Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A