NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ971383
Record Type: Journal
Publication Date: 2012-Apr
Pages: 19
Abstractor: As Provided
ISSN: ISSN-0265-5322
The Contribution of Test-Takers' Speech Content to Scores on an English Oral Proficiency Test
Sato, Takanori
Language Testing, v29 n2 p223-241 Apr 2012
The content that test-takers attempt to convey is not always included in the construct definition of "general" English oral proficiency tests, although some English-for-academic-purposes (EAP) speaking tests and most writing tests tend to place great emphasis on the evaluation of the content or ideas in the performance. This study investigated the relative contribution of linguistic criteria and the elaboration of speech content to scores on a test of speaking proficiency. A speaking test was designed and administered to Japanese undergraduates to determine what criteria English teachers associate with general oral proficiency. Nine raters were recruited to rate 30 students' monologues on three topics, using intuitive judgments of oral proficiency (referred to as "Overall communicative effectiveness"). Following this, they assigned scores to the monologues using five criteria: "Grammatical accuracy", "Fluency", "Vocabulary range", "Pronunciation", and "Content elaboration/development". The raters were also asked to provide open-ended written comments on the factors contributing to their intuitive judgments. Statistical analyses of the scores--Rasch measurement, multiple regression, and multivariate generalizability (G) theory analysis--revealed that "Content elaboration/development" made a substantive contribution to the intuitive judgments and composite score. The present study enriches our understanding of general oral proficiency and the construct definition of proficiency tests. (Contains 1 note and 6 tables.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Japan