NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ779256
Record Type: Journal
Publication Date: 2007
Pages: 36
Abstractor: Author
Reference Count: 69
ISSN: ISSN-0265-5322
Construct Validation of Analytic Rating Scales in a Speaking Assessment: Reporting a Score Profile and a Composite
Sawaki, Yasuyo
Language Testing, v24 n3 p355-390 2007
This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score. Confirmatory factor analysis (CFA) and multivariate generalizability theory (G theory) were combined to analyze the responses of 214 admits to a study-abroad program to two role-play speaking tasks in a Spanish speaking assessment designed for student placement and diagnosis. The CFA and G theory approaches provided complementary information, which generally confirmed the key features of the assessment design: (1) the multicomponential and yet highly correlated nature of the five analytic rating scales: Pronunciation, Vocabulary, Cohesion, Organization and Grammar, (2) the high dependability of the ratings and the resulting placement decisions appropriate for the high-stakes decision-making context based on these analytic rating scales and the composite score, and (3) the largest contribution of Grammar to the composite score variance, which was consistent with the intention of program faculty members to reflect in the test design the relative importance of knowledge of grammar for students' academic success in the study-abroad program. (Contains 10 tables, 2 figures, and 1 footnote. Appended are: LAAS [Language Ability Assessment System] Spanish Speaking Test: Ratings of Language Ability in Speaking; and Study Universe-Score and Absolute-Error Variance-Covariance Component Estimates.) [An earlier version of this paper was presented at the Annual Language Testing Research Colloquium (24th, Hong Kong, December 2002).]
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A