ERIC Number: ED255543
Record Type: RIE
Publication Date: 1985-Apr-1
Reference Count: 0
Relationships between Direct and Indirect Measures of Writing Ability.
Carlson, Sybil B.; Camp, Roberta
This paper reports on Educational Testing Service research studies investigating the parameters critical to reliability and validity in both the direct and indirect writing ability assessment of higher education applicants. The studies involved: (1) formulating an operational definition of writing competence; (2) designing and pretesting writing assessment instruments; (3) identifying and controlling parameters influencing a writing assessment program; and (4) the scoring and interpretation of measurement outcomes. Data from 638 college applicants representing three foreign language groups plus a small sample of native English speakers were collected from four writing samples scored by several methods, Test of English as a Foreign Language (TOEFL) scores, and Graduate Record Examination (GRE) General test scores. Correlational and factor analyses revealed these key findings: (1) holistic scores, discourse-level scores, and sentence-level scores were so closely related that holistic scores alone should be sufficient; (2) correlations among topics were as high across topic types as within topic types; (3) similar scoring by different types of raters suggest substantial agreement in the standards used; (4) writing sample and TOEFL scores were highly related but each also measured other aspects of English proficiency; and (5) relationship patterns between holistic writing sample scores and item type scores within GRE General Test sections were consistent with other GRE studies. (Author/BS)
Descriptors: College Entrance Examinations, Computer Assisted Testing, English (Second Language), Essay Tests, Experimenter Characteristics, Higher Education, Holistic Evaluation, Interrater Reliability, Language Proficiency, Scoring, Test Construction, Test Reliability, Test Validity, Writing Evaluation, Writing Skills
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: Graduate Record Examinations; Test of English as a Foreign Language; Writers Workbench (Computer Software)
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (Chicago, IL, March 31-April 4, 1985).