NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ776029
Record Type: Journal
Publication Date: 2007
Pages: 36
Abstractor: Author
Reference Count: 48
ISSN: ISSN-0265-5322
Evaluating Analytic Scoring for the TOEFL[R] Academic Speaking Test (TAST) for Operational Use
Xi, Xiaoming
Language Testing, v24 n2 p251-286 2007
This study explores the utility of analytic scoring for TAST in providing useful and reliable diagnostic information for operational use in three aspects of candidates' performance: delivery, language use and topic development. One hundred and forty examinees' responses to six TAST tasks were scored analytically on these three aspects of speech. G studies were used to investigate the dependability of the analytic scores, the distinctness of the analytic dimensions, and the variability of analytic score profiles. Raters' perceptions of dimension separability were obtained using a questionnaire. It was found that the dependability of analytic scores averaged across six tasks and double ratings was acceptable for both operational and practice settings. However, scores averaged across two tasks and double ratings were not reliable enough for operational use. Correlations among the analytic scores by task were high but those between delivery and topic development were lower. These results were corroborated by raters' perceptions. When averaged across tasks or task types, correlations among the analytic scores were very high, and the profiles of scores were flat. The utility of analytic scoring is discussed, considering both score dependability and whether analytic scores provide diagnostic information beyond that provided by holistic scores. The following are appended: (1) Rating criteria used by the instructors; and (2) Descriptives of the analytic scores of the 140 sample. (Contains 10 tables, 3 figures and 3 footnotes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language