ERIC Number: ED175944
Record Type: RIE
Publication Date: 1979-Apr-9
Reference Count: 0
The Assessment of Writing Proficiency via Qualitative Ratings of Writing Samples.
Steele, Joe M.
The College Outcome Measures Project/American College Testing Program (COMP/ACT) Writing Assessment is described, and issues of validity and reliability in the assessment of writing samples using qualitative rating scales are explored. COMP/ACT is composed of three role-playing tasks in the social sciences, natural sciences, and arts, which are recorded on audiotapes. Evaluators are chosen for their expertise in written communication. Qualitative rating scales are used to define five levels of proficiency on three broad criteria: the degree to which the writer addresses the intended audience; the development and organization of oertinent information; and the skillful use of language. Results of several studies indicated that these scales are sensitive to various levels of proficiency and can be applied in a highly reliable manner. Increasing the number of writing samples per student to three significantly increased reliability, but increasing the number of raters beyond two did not greatly enhance reliability. Use of a refined set of training materials for raters resulted in even higher levels of reliability. (The use of analytic as well as holistic rating procedures was also explored). (Author/MH)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Interrater Reliability
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, California, April 9-11, 1979)