NotesFAQContact Us
Search Tips
ERIC Number: EJ916219
Record Type: Journal
Publication Date: 2011-Apr
Pages: 21
Abstractor: As Provided
Reference Count: 33
ISSN: ISSN-0033-6882
Understanding Discrepancies in Rater Judgement on National-Level Oral Examination Tasks
Ang-Aw, Hui Teng; Goh, Christine Chuen Meng
RELC Journal: A Journal of Language Teaching and Research, v42 n1 p31-51 Apr 2011
The oral examination is an important component of the high-stakes "O" Level examination in Singapore taken by 16-17 year olds whose first language may or may not be English. In spite of this, there has been sparse research into the examination. This paper reports findings of an exploratory study which attempted to determine whether there were any discrepancies in rater judgements and thereafter, explore the nature and scope of the discrepancies identified. Four audio recordings were obtained from a simulated oral examination of four candidates conducted by a trained "O" Level oral examiner. Seven other trained "O" Level oral examiners were asked to rate the recordings individually and provide concurrent verbal reports. Questionnaires were also given to the raters for data triangulation after the verbalization. The data were analysed through Verbal Protocol Analysis and descriptive statistics. Rater discrepancies detected in the scores were qualitatively determined to be due to four differences: emphases on factors assessed, constructs of oral proficiency, rater interpretations and approaches in assessment. These findings provide valuable insights into raters' perceptions of the construct of speaking and offer implications for rater training and the development of rating scales. (Contains 7 tables and 2 figures.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Singapore