ERIC Number: EJ1110017
Record Type: Journal
Publication Date: 2013-Dec
Pages: 125
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Investigating the Value of Section Scores for the "TOEFL iBT"® Test. "TOEFL iBT"® Research Report. TOEFL iBT-21. ETS Research Report RR-13-35
Sawaki, Yasuyo; Sinharay, Sandip
ETS Research Report Series, Dec 2013
This study investigates the value of reporting the reading, listening, speaking, and writing section scores for the "TOEFL iBT"® test, focusing on 4 related aspects of the psychometric quality of the TOEFL iBT section scores: reliability of the section scores, dimensionality of the test, presence of distinct score profiles, and the section scores' generalizability for norm-referenced decisions as well as the dependability of criterion-referenced decisions for international student admission. Four operational TOEFL iBT test forms were analyzed for all examinees as well as for 3 native language (L1) groups (Arabic, Korean, and Spanish). Haberman's (2008) subscore analysis suggested that the speaking section score had added value due to its relative distinctness from the other modalities. Consistent with the subscore analysis results, a series of exploratory factor analyses (EFAs) indicated the possibility of the presence of 2 correlated factors--a reading/listening/writing factor and a speaking factor. In contrast, the CFAs conducted separately for the 3 L1 groups as well as a multiple-group confirmatory factor analyses (CFAs) identified a correlated 4-factor model with reading, listening, speaking, and writing factors as the best representation of the structure of the entire test for all examinees as well as for the 3 L1 groups. Reliability of the observed section scores for norm-referenced score interpretations and the dependability of classification decisions made based on different cut scores were generally satisfactory while they were also found to be relatively low in some circumstances. Based on the mixed results concerning the value-added information the TOEFL iBT section scores provide, recommendations for future research directions and some key issues of consideration for high-stakes decision making based on the section scores were summarized.
Descriptors: Scores, Computer Assisted Testing, Factor Analysis, Correlation, Language Tests, Second Language Learning, English (Second Language), Psychometrics, Test Reliability, Profiles, Generalization, Criterion Referenced Tests, Foreign Students, College Admission, Native Language, Semitic Languages, Korean, Spanish, Oral Language, Norm Referenced Tests, High Stakes Tests, Value Added Models, Decision Making
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A