NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1112141
Record Type: Journal
Publication Date: 2016
Pages: 16
Abstractor: As Provided
ISSN: ISSN-1543-4303
Investigating Score Dependability in English/Chinese Interpreter Certification Performance Testing: A Generalizability Theory Approach
Han, Chao
Language Assessment Quarterly, v13 n3 p186-201 2016
As a property of test scores, reliability/dependability constitutes an important psychometric consideration, and it underpins the validity of measurement results. A review of interpreter certification performance tests (ICPTs) reveals that (a) although reliability/dependability checking has been recognized as an important concern, its theoretical importance overshadows the operational efforts to measure score reliability/dependability; and (b) although multiple sources of measurement error can contribute variability to total score variance, rater effects have been regarded as the only source of error and modeled through classical test theory in the form of interrater reliability coefficients. Therefore, the study was initiated to investigate score dependability for a rater-mediated assessment of English-to-Chinese simultaneous interpreting, using generalizability theory. The results show the following: (a) the information completeness (InfoCom) ratings were more dependable than the ratings of fluency of delivery (FluDel) and target-language quality (TLQual); (b) the addition of tasks was more effective in raising dependability for InfoCom than the use of extra raters, but the effect was reversed for FluDel and TLQual; and (c) three different weighting schemes produced small variations in the composite-score dependability, but the InfoCom rating scale accounted for the largest proportions of the composite universe-score variances. These results are discussed for English/Chinese ICPTs.
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: China (Beijing)
Grant or Contract Numbers: N/A