NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1109652
Record Type: Journal
Publication Date: 2015-Jun
Pages: 16
Abstractor: As Provided
ISSN: EISSN-2330-8516
Automated Trait Scores for "TOEFL"® Writing Tasks. Research Report. ETS RR-15-14
Attali, Yigal; Sinharay, Sandip
ETS Research Report Series, Jun 2015
The "e-rater"® automated essay scoring system is used operationally in the scoring of "TOEFL iBT"® independent and integrated tasks. In this study we explored the psychometric added value of reporting four trait scores for each of these two tasks, beyond the total e-rater score.The four trait scores are word choice, grammatical conventions, fluency and organization, and content. Trait scores were computed on the basis of several criteria for determining feature weights: regression parameters of the trait features on human scores, reliability of trait features, and coefficients of features from a principal component analysis. In addition, augmented trait scores, based on information from other trait scores, were also analyzed.The psychometric added value of trait scores beyond total e-rater scores was evaluated by comparing the ability to predict a particular trait score on one task from the same trait score on the other task versus the e-rater score on the other task. Results supported the use of trait scores, and are discussed in terms of their contribution to the construct validity of e-rater as an alternative essay scoring method.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A