NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ982997
Record Type: Journal
Publication Date: 2012-Oct
Pages: 23
Abstractor: As Provided
Reference Count: N/A
ISBN: N/A
ISSN: ISSN-1075-2935
Adapting CEF-Descriptors for Rating Purposes: Validation by a Combined Rater Training and Scale Revision Approach
Harsch, Claudia; Martin, Guido
Assessing Writing, v17 n4 p228-250 Oct 2012
We explore how a local rating scale can be based on the Common European Framework CEF-proficiency scales. As part of the scale validation (Alderson, 1991; Lumley, 2002), we examine which adaptations are needed to turn CEF-proficiency descriptors into a rating scale for a local context, and to establish a practicable method to revise the initial scale-draft while simultaneously training raters for trialling the rating scale. We propose a data-driven approach focusing on the descriptors, involving raters in the scale revision as part of the scale validation process (Barkaoui, 2010a; Cohen, 1994). Rating data were analysed during a two-month period, employing 19 writing tasks and 13 raters; individual descriptors and raters were monitored for consistency. Our approach facilitated identifying causes of low agreement, let it be raters or scale wording, which in turn allowed targeted feedback to raters and revisions of scale-wording where indicated. We conclude with a summary of necessary adaptations of CEF-descriptors for rating purposes, and we outline further steps to validate the revised rating scale. We aim to contribute to improve rater training and rating scale development based on the CEF, since the literature for "real" test development contexts seems rather scarce (Barkaoui, 2010b; Knoch, 2011a,b). (Contains 10 tables and 1 figure.)
Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A