NotesFAQContact Us
Search Tips
Back to results
ERIC Number: ED505560
Record Type: Non-Journal
Publication Date: 2009-Feb
Pages: 93
Abstractor: As Provided
ISSN: ISSN-1930-9317
Criterion-Related Validity of the TOEFL iBT Listening Section. TOEFL iBT Research Report. RR-09-02
Sawaki, Yasuyo; Nissan, Susan
Educational Testing Service
The study investigated the criterion-related validity of the "Test of English as a Foreign Language"[TM] Internet-based test (TOEFL[R] iBT) Listening section by examining its relationship to a criterion measure designed to reflect language-use tasks that university students encounter in everyday academic life: listening to academic lectures. The design of the criterion measure was informed by students' responses to a survey on the frequency and importance of various classroom tasks that require academic listening, and the relationship of these tasks to successful course completion. The criterion measure consisted of three videotaped lectures (in physics, history, and psychology) and included tasks created by content experts who are former university professors of the relevant content area. These tasks reflected what the content experts expected students to have comprehended during the lecture. The criterion measure and the TOEFL iBT Listening section were administered to nonnative speakers of English who were enrolled in undergraduate and graduate programs. Data from 221 participants were analyzed. Substantial correlations were observed between the criterion measure and the TOEFL iBT Listening section score for the entire sample and for subgroups (Pearson correlation coefficients ranging from .56 to .74 and disattenuated correlations ranging from .62 to .82). Moreover, the analysis of the mean scores on the criterion measure for different ability groups indicated that participants who scored at or above typical cut scores for international student admission to academic programs (i.e., TOEFL iBT Listening section score of 14 or above) scored, on average, nearly 50% or more on the criterion measure, demonstrating reasonable comprehension of the academic lectures. Ten appendices are included: (1) Survey Instrument for the University Student Survey; (2) Incidence of Various Activities Requiring Student Listening in the Classroom (N = 136); (3) Importance of Various Activities Requiring Student Listening in the Classroom (N = 136); (4) Frequency of Course Assignments and Their Percentages Composed of Final Course Grade (N = 136); (5) Information Source for Completing Course Assignments (N = 136); (6) Characteristics of Video Lectures; (7) Physics Video Exercise; (8) History Video Exercise; (9) Psychology Video Exercise; and (10) Questions on Student Familiarity With Lecture Content (Sample Taken From the Psychology Video Set). (Contains 9 tables, 8 figures and 15 notes.
Educational Testing Service. Rosedale Road Mailstop 19R, Princeton, NJ 08541-0001. Tel: 609-921-9000; Fax: 609-734-5410; Web site:
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Educational Testing Service
Identifiers - Location: California; Canada; Connecticut; Georgia; Indiana; Iowa; Michigan; Wisconsin
Identifiers - Assessments and Surveys: Test of English as a Foreign Language