NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ994828
Record Type: Journal
Publication Date: 2013
Pages: 18
Abstractor: As Provided
Reference Count: 53
ISBN: N/A
ISSN: ISSN-0895-7347
Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings
Kuhlemeier, Hans; Hemker, Bas; van den Bergh, Huub
Applied Measurement in Education, v26 n1 p16-33 2013
In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers' ratings of their own candidates' performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged [plus or minus]16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers' performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. (Contains 2 tables.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Netherlands