ERIC Number: EJ1109294
Record Type: Journal
Publication Date: 2014-Jun
Pages: 25
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Using Writing Process and Product Features to Assess Writing Quality and Explore How Those Features Relate to Other Literacy Tasks. Research Report. ETS RR-14-03
Deane, Paul
ETS Research Report Series, Jun 2014
This paper explores automated methods for measuring features of student writing and determining their relationship to writing quality and other features of literacy, such as reading rest scores. In particular, it uses the "e-rater"™ automatic essay scoring system to measure "product" features (measurable traits of the final written text) and features extracted from keystroke logs to measure process features (measurable features of the writing process). These techniques are applied to student essays written during large-scale pilot administrations of writing assessments developed for ETS's "CBAL"™ research initiative. The design makes it possible to explore the factor structures of these product and process features and to examine how well they generalize beyond a single test session to predict underlying traits such as writing ability and reading level. Three product factors are identified, connected to fluency, accuracy, and content. The process factors are identified, corresponding to hesitancy behaviors, editing behaviors, and burst span (the extent to which text is produced in long bursts with only short internal pauses). The results suggest that writing process and product features have stable factor structures that generalize to predict writing quality, though there are some genre- or task-specific differences.
Descriptors: Writing Processes, Writing Evaluation, Student Evaluation, Writing Skills, Writing Tests, Essay Tests, Computer Assisted Testing, Pilot Projects, Factor Structure, Factor Analysis, Generalization, Writing Ability, Reading, Accuracy, Editing, Grade 8, Middle School Students, Scores, Prompting, Correlation, Regression (Statistics)
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 8; Junior High Schools; Middle Schools; Elementary Education; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A