NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1120112
Record Type: Journal
Publication Date: 2016
Pages: 18
Abstractor: As Provided
ISSN: ISSN-1543-4303
Automated Essay Evaluation for English Language Learners: A Case Study of "MY Access"
Hoang, Giang Thi Linh; Kunnan, Antony John
Language Assessment Quarterly, v13 n4 p359-376 2016
Computer technology made its way into writing instruction and assessment with spelling and grammar checkers decades ago, but more recently it has done so with automated essay evaluation (AEE) and diagnostic feedback. And although many programs and tools have been developed in the last decade, not enough research has been conducted to support or evaluate the claims of the developers. This study examined the effectiveness of automated writing instructional programs in consistent scoring of essays and appropriate feedback to student writers. It examined the scoring and instructional program called "MY Access! Home Edition," which has an error feedback tool called "My Editor" to address these issues. The study combined a quantitative study of agreement and correlational analyses with an analysis of content and topic. Participants included 114 English language learners who wrote 147 essays to three writing prompts, which were graded by trained EFL raters and "MY Access." From the sample, 15 randomly selected essays were also used for an error analysis comparing "My Editor" with human annotations to examine "My Editor"'s accuracy. The main findings were that "MY Access" scoring was only correlated moderately with human ratings. Furthermore, because "MY Access" scoring is limited to the recognition of content words, not how these words are organized at the discourse level, it did not detect slightly off-topic essays and plagiarism. Finally, "My Editor"'s error feedback, with 73% precision and 30% recall, did not meet the expectations of an accurate tool. In conclusion, the home edition of "MY Access" was not found to be useful as an independent instructional tool. These findings give us pause regarding the effectiveness of "MY Access."
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California; Vietnam
Grant or Contract Numbers: N/A