NotesFAQContact Us
Search Tips
ERIC Number: ED432589
Record Type: Non-Journal
Publication Date: 1999-Apr-21
Pages: 30
Abstractor: N/A
Reference Count: N/A
Trait Ratings for Automated Essay Grading.
Shermis, Mark D.; Koch, Chantal Mees; Page, Ellis B.; Keith, Timothy Z.; Harrington, Susanmarie
This study used Project Essay Grade (PEG) to evaluate essays both holistically and with the rating of traits (content, organization, style, mechanics, and creativity) for Web-based student essays that serve as placement tests at a large Midwestern university. In addition, the use of a TopicScore, or measure of topic content for each assignment, was incorporated into the PEG model to determine how well it would correlate with the five traits. In the first experiment, essays from 500 students were used to create statistical predictions for the PEG software. In the second experiment, the ratings from 300 essays were compared with ratings from 6 human judges. The interjudge correlation of the raters was only 0.51, but the prediction of all 6 judges, in the blind test, reached 0.83 for the PEG program. Of the five traits, "content" and "creativity" had the highest interjudge correlations. The new TopicScore correlated most highly with content, providing some measure of PEG's construct validity. The PEG software was an efficient means of grading the essays, with approximately six documents graded per second. (Contains 3 figures, 6 tables, and 15 references.) (Author/SLD)
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A