NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ863234
Record Type: Journal
Publication Date: 2009
Pages: 16
Abstractor: As Provided
Reference Count: 15
ISSN: ISSN-0013-1644
Validity of Scores for a Developmental Writing Scale Based on Automated Scoring
Attali, Yigal; Powers, Donald
Educational and Psychological Measurement, v69 n6 p978-993 2009
A developmental writing scale for timed essay-writing performance was created on the basis of automatically computed indicators of writing fluency, word choice, and conventions of standard written English. In a large-scale data collection effort that involved a national sample of more than 12,000 students from 4th, 6th, 8th, 10th, and 12th grade, students wrote (in 30-min sessions) up to four essays in two modes of writing on topics selected from a pool of 20 topics. Scale scores were created by combining essay indicators in a standard way to compute essay scores that shared the same scoring standards across essay prompts and student grade levels. A series of ancillary analyses and studies were conducted to examine the validity of scale scores. Cross-classified random effects modeling of scores confirmed that the particular prompts on which essays are written have little effect on scores. The reliability of scores was found to be higher compared to previous reliability estimates of human essay scores. A human scoring experiment confirmed that the developmental sensitivity of scale scores and human scores was similar. A longitudinal study confirmed the expected gains in scores over a 1-year period. (Contains 7 tables.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 10; Grade 12; Grade 4; Grade 6; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A