NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ893471
Record Type: Journal
Publication Date: 2010-Jul
Pages: 19
Abstractor: As Provided
Reference Count: 34
ISSN: ISSN-0265-5322
Validation of Automated Scores of TOEFL iBT Tasks against Non-Test Indicators of Writing Ability
Weigle, Sara Cushing
Language Testing, v27 n3 p335-353 Jul 2010
Automated scoring has the potential to dramatically reduce the time and costs associated with the assessment of complex skills such as writing, but its use must be validated against a variety of criteria for it to be accepted by test users and stakeholders. This study approaches validity by comparing human and automated scores on responses to TOEFL[R] iBT Independent writing tasks with several non-test indicators of writing ability: student self-assessment, instructor assessment, and independent ratings of non-test writing samples. Automated scores were produced using "e-rater"[R], developed by Educational Testing Service (ETS). Correlations between both human and "e-rater" scores and non-test indicators were moderate but consistent, providing criterion-related validity evidence for the use of "e-rater" along with human scores. The implications of the findings for the validity of automated scores are discussed. (Contains 10 tables and 2 notes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A