NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1083348
Record Type: Journal
Publication Date: 2015
Pages: 30
Abstractor: As Provided
ISSN: ISSN-1543-4303
What Are We Aligning Tests to When We Report Test Alignment to the CEFR?
Harsch, Claudia; Hartig, Johannes
Language Assessment Quarterly, v12 n4 p333-362 2015
The study reported here investigates the validity of judgments made when aligning tests to the "Common European Framework of Reference" (CEFR). Listening tests operationalizing pre-defined difficulty-determining characteristics were to be aligned to CEFR levels. We employed a modified version of the item-descriptor-matching-method. Ten judges stated the CEFR descriptors they thought each item operationalizes and assigned a global CEFR level per item. We compared agreement on CEFR level judgments and CEFR descriptors quoted. Analyzing the relationship between CEFR level judgments and item ratings of difficulty-determining characteristics shed light on further criteria the judges employed. Follow-up interviews helped to triangulate the findings by examining judges' perceptions of the alignment procedure. We found that judges relied on different criteria and CEFR descriptors to a varying degree, interpreting CEFR levels differently. There seemed little comparability in what aspects judges used to form their global CEFR judgments. Therefore, if an alignment does not take into account the meaning of the CEFR levels as expressed by existing CEFR descriptors, this raises issues with alignment validity, and hence the validity of test-score interpretation and usage. Given the impact of using CEFR aligned tests for high-stakes purposes, this article aims to shed more light on what assigning a CEFR level to a test actually means.
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Germany
Grant or Contract Numbers: N/A