ERIC Number: ED212659
Record Type: RIE
Publication Date: 1978-Nov
Reference Count: 0
The Effects of Differing Response Criteria on the Assessment of Writing Competence.
The purpose of this study was to investigate the relative validities of four essay scoring systems, reflecting alternative conceptualizations of the writing process, for identifying "competent" writers. Each rater was trained in two of the four scoring systems: General Impression Scoring (GI), Diederich Expository Scale (DES), CSE Analytic Scale (CSE), or T-Unit Analysis (T). Writing samples were randomly selected from four composition classes: low performance high school writers, high performance high school writers, low performance college writers, and high performance college writers. The experimental design was a hierarchical mixed model: all subjects were crossed with all raters, while topics and scoring systems were nested within raters. All four scoring systems had high reliability. Different scoring systems produced different patterns of results, which may mean that the writing performance of each group cannot be adequately described by the same scoring system. Three of the systems (GI, DES, and CSE) were associated with some differences in group performance; T-Unit was not, making it virtually useless for classification purposes. The classification accuracy of the three systems (GI, DES, CSE) was questionable in distinguishing all but the high school low group. (BW)
Publication Type: Reports - Research; Tests/Questionnaires
Education Level: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC.
Authoring Institution: California Univ., Los Angeles. Center for the Study of Evaluation.
Identifiers: CSE Analytic Scale; Diederich Rating Scale for Essays; General Impression Scoring; Inter Rater Reliability; T Units