NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ895962
Record Type: Journal
Publication Date: 2010-Aug
Pages: 17
Abstractor: As Provided
Reference Count: 17
ISBN: N/A
ISSN: ISSN-1540-2525
Performance of a Generic Approach in Automated Essay Scoring
Attali, Yigal; Bridgeman, Brent; Trapani, Catherine
Journal of Technology, Learning, and Assessment, v10 n3 Aug 2010
A generic approach in automated essay scoring produces scores that have the same meaning across all prompts, existing or new, of a writing assessment. This is accomplished by using a single set of linguistic indicators (or features), a consistent way of combining and weighting these features into essay scores, and a focus on features that are not based on prompt-specific information or vocabulary. This approach has both logistical and validity-related advantages. This paper evaluates the performance of generic scores in the context of the e-rater[R] automated essay scoring system. Generic scores were compared with prompt-specific scores and scores that included prompt-specific vocabulary features. These comparisons were performed with large samples of essays written to three writing assessments: The GRE General Test argument and issue tasks and the TOEFL independent task. Criteria for evaluation included level of agreement with human scores, discrepancy from human scores across prompts, and correlations with other available scores. Results showed small differences between generic and prompt-specific scores and adequate performance of both types of scores compared to human performance. (Contains 3 tables.)
Technology and Assessment Study Collaborative. 332 Campion Hall, Boston College, Chestnut Hill, MA 02467. Tel: 617-552-4521; Fax: 617-552-8419; e-mail: jtal_editor@bc.edu; Web site: http://escholarship.bc.edu/jtla
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Graduate Record Examinations; Test of English as a Foreign Language