NotesFAQContact Us
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1109303
Record Type: Journal
Publication Date: 2015-Dec
Pages: 18
Abstractor: As Provided
ISSN: EISSN-2330-8516
Exploring the Feasibility of Using Writing Process Features to Assess Text Production Skills. Research Report. ETS RR-15-26
Deane, Paul; Zhang, Mo
ETS Research Report Series, Dec 2015
In this report, we examine the feasibility of characterizing writing performance using process features derived from a keystroke log. Using data derived from a set of "CBAL"™ writing assessments, we examine the following research questions: (a) How stable are the keystroke timing and process features across testing occasions?; (b) How consistent are the patterns of feature-human correlation across genres and topics?; (c) How accurately can we predict human ratings on writing fundamentals using a combination of the keystroke timing and process features, and what are the contributions of each feature to the reliable variance in the human ratings?; and (d) If we train a predictive model on one prompt, how well do its predictions generalize to the other prompts of the same or different genre? The results of the study indicate that keystroke log features vary considerably in stability across testing occasions and display somewhat different patterns of feature-human correlation across genres and topics. However, using the most stable features, we can obtain moderate to strong prediction of human essay scores, and those models generalize reasonably well across prompts though more strongly within than across writing genres.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 6; Intermediate Grades; Middle Schools; Elementary Education; Grade 7; Junior High Schools; Secondary Education; Grade 8; Grade 9; High Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Idaho
Grant or Contract Numbers: N/A