NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ921715
Record Type: Journal
Publication Date: 2011-Apr
Pages: 22
Abstractor: As Provided
Reference Count: 25
ISSN: ISSN-0265-5322
Investigating the Effectiveness of Individualized Feedback to Rating Behavior--A Longitudinal Study
Knoch, Ute
Language Testing, v28 n2 p179-200 Apr 2011
The effectiveness of individualized feedback on rater behavior has been investigated in several previous studies (e.g. Elder, Knoch, Barkhuizen, & von Randow, 2005; Lunt, Morton, & Wigglesworth, 1994; O'Sullivan & Rignall, 2007; Wigglesworth, 1993). The findings of these studies are somewhat inconclusive. However, all of these studies investigated the value of such feedback on a one-off basis, and it is thus not clear how raters use this type of feedback over several administrations of a test. Furthermore, previous research has focused only on one language skill at a time, and we do not know if raters can incorporate the feedback in a similar way when rating speaking and writing. This study tracks the rating behavior of 19 raters assessing a large-scale English for specific purposes (ESP) assessment for the health professions over eight administrations. After each administration, raters received detailed performance profiles of their rating behavior which were generated using many-faceted Rasch measurement. Raters also completed a questionnaire and a subset of raters were interviewed to ascertain their views on the effectiveness of this feedback. The findings showed that the raters were rating no better when receiving the feedback than when they had not received feedback and neither speaking nor writing raters were able to incorporate the feedback more successfully. Although the raters were generally positive about the success of the feedback, there was no relationship between their perceptions of the feedback and the success of the feedback. (Contains 6 figures, 12 tables and 1 note.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A