NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1142406
Record Type: Journal
Publication Date: 2017-Jun
Pages: 24
Abstractor: As Provided
ISSN: ISSN-0922-4777
Writing Evaluation: Rater and Task Effects on the Reliability of Writing Scores for Children in Grades 3 and 4
Kim, Young-Suk Grace; Schatschneider, Christopher; Wanzek, Jeanne; Gatlin, Brandy; Al Otaiba, Stephanie
Reading and Writing: An Interdisciplinary Journal, v30 n6 p1287-1310 Jun 2017
We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of 0.90 and 0.80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written compositions were evaluated in widely used evaluation methods for developing writers: holistic scoring, productivity, and curriculum-based writing scores. Results showed that 54 and 52% of variance in narrative and expository compositions were attributable to true individual differences in writing. Students' scores varied largely by tasks (30.44 and 28.61% of variance), but not by raters. To reach the reliability of 0.90, multiple tasks and raters were needed, and for the reliability of 0.80, a single rater and multiple tasks were needed. These findings offer important implications about reliably evaluating children's writing skills, given that writing is typically evaluated by a single task and a single rater in classrooms and even in some state accountability systems.
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Grade 3; Primary Education; Early Childhood Education; Grade 4; Intermediate Grades
Audience: N/A
Language: English
Sponsor: Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) (NIH)
Authoring Institution: N/A
Grant or Contract Numbers: P50HD052120