NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1088021
Record Type: Journal
Publication Date: 2016-Jan
Pages: 17
Abstractor: As Provided
ISSN: ISSN-0265-5322
A Comparison of Newly-Trained and Experienced Raters on a Standardized Writing Assessment
Attali, Yigal
Language Testing, v33 n1 p99-115 Jan 2016
A short training program for evaluating responses to an essay writing task consisted of scoring 20 training essays with immediate feedback about the correct score. The same scoring session also served as a certification test for trainees. Participants with little or no previous rating experience completed this session and 14 trainees who passed an accuracy threshold proceeded to score other essays. Performance of the newly-trained raters was compared to that of 16 expert raters with extensive experience in scoring responses to the writing task. Results showed that the scores from the newly-trained group of raters exhibited similar measurement properties (mean and variability of scores, reliability and various validity coefficients, and underlying factor structure) to those from the experienced group of raters. Implications for the place of initial training and screening of raters on rater performance are discussed.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A