NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1143726
Record Type: Journal
Publication Date: 2016
Pages: 21
Abstractor: As Provided
ISSN: EISSN-2056-9017
Investigating the Application of Automated Writing Evaluation to Chinese Undergraduate English Majors: A Case Study of "WriteToLearn"
Liu, Sha; Kunnan, Antony John
CALICO Journal, v33 n1 p71-91 2016
This study investigated the application of "WriteToLearn" on Chinese undergraduate English majors' essays in terms of its scoring ability and the accuracy of its error feedback. Participants were 163 second-year English majors from a university located in Sichuan province who wrote 326 essays from two writing prompts. Each paper was marked by four human raters as well as "WriteToLearn." Many-facet Rasch measurement (MFRM) was conducted to calibrate "WriteToLearn"'s rating performance in scoring the whole set of essays against those of four trained human raters. In addition, the accuracy of WriteToLearn's feedback on 60 randomly selected essays was compared with the feedback provided by human raters. The two main findings related to scoring were that: (1) "WriteToLearn" was more consistent but highly stringent when compared to the four trained human raters in scoring essays; and (2) "WriteToLearn" failed to score seven essays. In terms of error feedback, "WriteToLearn" had an overall precision and recall of 49% and 18.7% respectively. These figures did not meet the minimum threshold of 90% precision (set by Burstein, Chodorow, and Leacock, 2003) for it to be considered a reliable error detecting tool. Furthermore, it had difficulty in identifying errors made by Chinese undergraduate English majors in the use of articles, prepositions, word choice and expression.
Equinox Publishing Ltd. Office 415, The Workstation, 15 Paternoster Row, Sheffield, S1 2BX, UK. Tel: +44-114-221-0285; Fax: +44-114-279-6522; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: China
Grant or Contract Numbers: N/A