NotesFAQContact Us
Search Tips
ERIC Number: ED307321
Record Type: Non-Journal
Publication Date: 1989-Mar
Pages: 41
Abstractor: N/A
Reference Count: N/A
Decentralized Large Scale Essay Scoring: Methods for Establishing and Evaluating Score Scale Stability and Reading Reliability.
Auchter, Joan Chikos; Patience, Wayne
The methods used by the General Educational Development Testing Service (GEDTS) to establish and maintain score stability and reading reliability on its direct assessment of writing are described. Using the 1988 site certification and monitoring results of several scoring sites, the focus is on describing how the score scale was established and how reader training and certification, scoring site certification, and scoring monitoring procedures were developed and implemented. The Tests of General Educational Development (GED) are used extensively to award high school equivalency diplomas. A writing essay was added in the 1988 administration. The nature of the tests required that essays be scored in decentralized scoring sessions. Holistic scoring with two readers was chosen, with a third reading if scores differed too much. The GED direct assessment is norm-referenced, calling for a descriptive, rather than prescriptive, scoring guide. Chief readers and readers receive training in scoring, and a site is only certified when readers have been trained. Systematic monitoring reduces or eliminates scoring drift. Approximately 150 persons attended the first Chief Reader training. At the end of the first year of site certification, 52 operational scoring sites were established. Monitoring results of 15 sites are presented. Ensuring fair and stable scoring from site to site requires careful planning and systematic procedures. The process so far illustrates the importance of training readers and monitoring the sites. Seven tables present monitoring results. Nine appendices provide additional information about the development process, including scoring, test instructions, and reader requirements. (SLD)
Publication Type: Speeches/Meeting Papers; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: General Educational Development Tests