NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ962949
Record Type: Journal
Publication Date: 2012
Pages: 16
Abstractor: As Provided
Reference Count: 47
ISSN: ISSN-0895-7347
Rater Language Background as a Source of Measurement Error in the Testing of English Language Learners
Kachchaf, Rachel; Solano-Flores, Guillermo
Applied Measurement in Education, v25 n2 p162-177 2012
We examined how rater language background affects the scoring of short-answer, open-ended test items in the assessment of English language learners (ELLs). Four native English and four native Spanish-speaking certified bilingual teachers scored 107 responses of fourth- and fifth-grade Spanish-speaking ELLs to mathematics items administered in English and Spanish. We examined the mean scores given by raters of different language backgrounds. Also, using generalizability theory, we examined the amount of score variation due to student (the object of measurement) and four sources of measurement error-item, language of testing, rater language background, and rater nested in rater language background. We observed a small, statistically significant difference between mean scores given by raters of different language background and a negligible score variation due to the main and interaction effect of rater. Provided that they are certified bilingual teachers, and regardless of language of testing, raters of different language backgrounds can score ELL responses to short-answer, open-ended items with comparable reliability. (Contains 4 tables, 1 figure, and 5 footnotes.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 4; Grade 5
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress