ERIC Number: EJ1219448
Record Type: Journal
Publication Date: 2019-Jul
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0265-5322
EISSN: N/A
A Generalizability Theory Study of Optimal Measurement Design for a Summative Assessment of English/Chinese Consecutive Interpreting
Language Testing, v36 n3 p419-438 Jul 2019
Summative assessment of interpretation is widely conducted in interpreting courses/programs to inform high-stakes decision making, such as the selection, certification, and conferral of academic degrees. Yet there has been very limited empirical research to investigate the score dependability of summative interpretation assessment. The present study therefore sets out to explore the optimal measurement design(s) for a locally created summative assessment of English/Chinese consecutive interpretation, based on multiple fully crossed generalizability studies. Major findings include the following: (a) overall the raters behaved more consistently by using the information completeness (InfoCom) scale rather than the fluency of delivery (FluDel) or target language quality (TLQual) scales; (b) the raters displayed greater variability in evaluating the Chinese-to-English interpretation rather than the English-to-Chinese interpretation; (c) although adding tasks worked more effectively in raising score dependability than using additional raters for the InfoCom ratings in the English-to-Chinese interpretation, the pattern was reversed for the other observations; and (d) two potentially optimal designs were identified for the English-to-Chinese direction, and one design for the other direction. These results are discussed, highlighting the complex nature of relationships among the assessment criterion, the interpreting directionality, the raters' dominant language and score dependability, together with the need to ensure score dependability for summative interpretation assessment.
Descriptors: Generalization, Decision Making, Summative Evaluation, Evaluators, Language Fluency, Chinese, English, English (Second Language), Scores, Correlation, Translation, Measurement, Language Tests, Program Development, Second Language Learning, Second Language Instruction, Foreign Countries, Undergraduate Students, Task Analysis, Evaluation Criteria
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: China
Grant or Contract Numbers: N/A