ERIC Number: EJ1111373
Record Type: Journal
Publication Date: 2005-Apr
Pages: 170
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
An Examination of Rater Orientations and Test-Taker Performance on English-for-Academic-Purposes Speaking Tasks. TOEFL® Monograph Series. MS-29. ETS RR-05-05
Brown, Annie; Iwashita, Noriko; McNamara, Tim
ETS Research Report Series, Apr 2005
This report documents two coordinated exploratory studies into the nature of oral English-for-academic-purposes (EAP) proficiency. Study I used verbal-report methodology to examine field experts? rating orientations, and Study II investigated the quality of test-taker discourse on two different Test of English as a Foreign Language? (TOEFL®) task types (independent and integrated) at different levels of proficiency. Study I showed that, with no guidance, domain experts distinguished and described qualitatively different performances using a common set of criteria very similar to those included in draft rating scales developed for the tasks at ETS. Study II provided empirical support for the criteria applied by the judges. The findings indicate that raters take a range of performance features into account within each conceptual category and that holistic ratings are driven by all of the assessment categories rather than, as has been suggested in earlier studies, predominantly by grammar.
Descriptors: Evaluators, English (Second Language), Language Tests, Second Language Learning, English for Academic Purposes, Grammar, Oral Language, Language Proficiency, Specialists, Rating Scales, Evaluation Criteria, Holistic Approach, Discourse Analysis, Verbal Communication, Phonology, Language Fluency, Pronunciation, Task Analysis, Foreign Countries, Statistical Analysis
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A