ERIC Number: EJ1247747
Record Type: Journal
Publication Date: 2020-Apr
Pages: 26
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0265-5322
EISSN: N/A
Evaluating Subscore Uses across Multiple Levels: A Case of Reading and Listening Subscores for Young EFL Learners
Choi, Ikkyu; Papageorgiou, Spiros
Language Testing, v37 n2 p254-279 Apr 2020
Stakeholders of language tests are often interested in subscores. However, reporting a subscore is not always justified; a subscore should provide reliable and distinct information to be worth reporting. When a subscore is used for decisions across multiple levels (e.g., individual test takers and schools), it needs to be justified for its reliability and distinctiveness at every relevant level. In this study, we examined whether reporting seven Reading and Listening subscores of the "TOEFL Primary"® test, a standardized English proficiency test for young English as a foreign language learners, could be justified for reporting at individual and school levels. We analyzed data collected in pilot administrations, in which 4776 students from 51 schools participated. We employed the classical test theory (CTT) based approaches of Haberman (2008) and Haberman, Sinharay, and Puhan (2009) for the individual and school-level investigations, respectively. We also supplemented the CTT-based approaches with a factor analytic approach for the individual level analysis and a multilevel modeling approach for the school-level analysis. The results differed across the two levels: we found little support for reporting the subscores at the individual level, but strong evidence supporting the added-value of the school-level subscores when the sample size for each school exceeds 50.
Descriptors: English (Second Language), Language Tests, Second Language Learning, Scores, Decision Making, Item Response Theory, Language Proficiency, Listening Comprehension Tests, Factor Analysis, Reading Tests, Institutional Characteristics, Test Reliability, Second Language Instruction, Foreign Countries, Psychometrics, Correlation, Profiles, Elementary School Students
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Kenya
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A