ERIC Number: EJ884646
Record Type: Journal
Publication Date: 2010-Jul
Pages: 27
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0142-6001
EISSN: N/A
Toward Automated Multi-Trait Scoring of Essays: Investigating Links among Holistic, Analytic, and Text Feature Scores
Lee, Yong-Won; Gentile, Claudia; Kantor, Robert
Applied Linguistics, v31 n3 p391-417 Jul 2010
The main purpose of the study was to investigate the distinctness and reliability of analytic (or multi-trait) rating dimensions and their relationships to holistic scores and "e-rater"[R] essay feature variables in the context of the TOEFL[R] computer-based test (TOEFL CBT) writing assessment. Data analyzed in the study were holistic and multi-trait essay scores provided by human raters and essay feature variable scores computed by "e-rater"-[R] (version 2.0) for two TOEFL CBT writing prompts. It was found that (i) all of the six multi-trait scores were not only correlated among themselves but also correlated with the holistic score, (ii) high correlations obtained among holistic and multi-trait scores were largely attributable to the impact of essay length on both holistic and multi-trait scoring, and (iii) some strong associations were confirmed between several "e-rater" variables and multi-trait rating dimensions. Implications are discussed for improving the multi-trait scoring of essays, refining "e-rater" essay feature variables, and validating automated essay scores.
Descriptors: Writing Evaluation, Writing Tests, Scoring, Essays, Computer Assisted Testing, English (Second Language), Second Language Learning, Language Tests, Scores, Computer Software, Correlation, Holistic Approach, Reliability
Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: jnls.cust.serv@oxfordjournals.org; Web site: http://applij.oxfordjournals.org/
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A