NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1168733
Record Type: Journal
Publication Date: 2017-Dec
Pages: 17
Abstractor: As Provided
ISSN: EISSN-2330-8516
Toward the Automated Scoring of Written Arguments: Developing an Innovative Approach for Annotation. Research Report. ETS RR-17-11
Song, Yi; Deane, Paul; Beigman Klebanov, Beata
ETS Research Report Series, Dec 2017
This project focuses on laying the foundations for automated analysis of argumentation schemes, supporting identification and classification of the arguments being made in a text, for the purpose of scoring the quality of written analyses of arguments. We developed annotation protocols for 20 argument prompts from a college-level test under the framework of the theory of argumentation schemes, which defines reasoning patterns in argumentation. The annotation protocols listed critical questions associated with each argumentation scheme, which makes the argument structure in a text explicit and classifiable. Furthermore, we annotated 200 student essays across four selected argument prompts to test if the annotation protocols can be applied reliably by human annotators. Preliminary results indicate that this method of analyzing argument structure is reliable and promising.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A