NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1111373
Record Type: Journal
Publication Date: 2005-Apr
Pages: 170
Abstractor: As Provided
ISSN: EISSN-2330-8516
An Examination of Rater Orientations and Test-Taker Performance on English-for-Academic-Purposes Speaking Tasks. TOEFL® Monograph Series. MS-29. ETS RR-05-05
Brown, Annie; Iwashita, Noriko; McNamara, Tim
ETS Research Report Series, Apr 2005
This report documents two coordinated exploratory studies into the nature of oral English-for-academic-purposes (EAP) proficiency. Study I used verbal-report methodology to examine field experts? rating orientations, and Study II investigated the quality of test-taker discourse on two different Test of English as a Foreign Language? (TOEFL®) task types (independent and integrated) at different levels of proficiency. Study I showed that, with no guidance, domain experts distinguished and described qualitatively different performances using a common set of criteria very similar to those included in draft rating scales developed for the tasks at ETS. Study II provided empirical support for the criteria applied by the judges. The findings indicate that raters take a range of performance features into account within each conceptual category and that holistic ratings are driven by all of the assessment categories rather than, as has been suggested in earlier studies, predominantly by grammar.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A