ERIC Number: ED555699
Record Type: Non-Journal
Publication Date: 2013-Dec
Abstractor: As Provided
Reference Count: 36
Automatically Scoring Short Essays for Content. CRESST Report 836
Kerr, Deirdre; Mousavi, Hamid; Iseli, Markus R.
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
The Common Core assessments emphasize short essay constructed response items over multiple choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way is found to score them automatically. Current automatic essay scoring techniques are inappropriate for scoring the content of an essay because they rely on either grammatical measures of quality or machine learning techniques, neither of which identifies statements of meaning (propositions) in the text. In this report, we explain our process of (1) extracting meaning from student essays in the form of propositions using our text mining framework called SemScape, (2) using the propositions to score the essays, and (3) testing our system's performance on two separate sets of essays. Results demonstrate the potential of this purely semantic process and indicate that the system can accurately extract propositions from student short essays, approaching or exceeding standard benchmarks for scoring performance. Five appendices include: (1) Concepts in Hearing and Vision Ontologies; (2) Hearing Essays; (3) Vision Essays; (4) Human and Computer Scores for Hearing Essays; and (5) Human and Computer Scores for Vision Essays.
Descriptors: Automation, Scoring, Essay Tests, Natural Language Processing, Semantics, Accuracy, Grade 4, Grade 5
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org
Publication Type: Reports - Research
Education Level: Grade 4; Intermediate Grades; Elementary Education; Grade 5; Middle Schools
Sponsor: Bill and Melinda Gates Foundation
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing