ERIC Number: EJ1111304
Record Type: Journal
Publication Date: 2008-Jan
Pages: 75
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Investigating the Criterion-Related Validity of the TOEFL® Speaking Scores for ITA Screening and Setting Standards for ITAS. TOEFL iBT Research Report. TOEFL iBT-03. ETS RR-08-02
Xi, Xiaoming
ETS Research Report Series, Jan 2008
Although the primary use of the speaking section of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT Speaking test) is to inform admissions decisions at English medium universities, it may also be useful as an initial screening measure for international teaching assistants (ITAs). This study provides criterion-related validity evidence for the use of TOEFL iBT Speaking for ITA screening and evaluates the effectiveness of using the scores for teaching assistantship (TA) assignment classification. Four universities participated in this study. Local ITA-screening tests or instructor recommendations were used as the criterion measures. Relationships between the TOEFL Speaking test and the local ITA tests were explored through observed and disattenuated correlations. These relationships were moderately strong, supporting the use of the TOEFL Speaking test for ITA screening. However, the strengths of the relationship between the TOEFL Speaking test and the local ITA tests were found to be somewhat different across universities depending on the extent to which the local test engaged and evaluated nonlanguage abilities. Implications of these findings are discussed. Binary and ordinal logistic regressions were used to investigate how effective TOEFL Speaking scores were in separating students into distinct TA assignment categories. At all four universities, TOEFL Speaking scores were significant predictors of students' TA assignments and were fairly accurate in classifying students for TA assignments. ROC curves were used to determine TOEFL Speaking cut scores for TA assignments at each university that would minimize false positives (i.e., true nonpasses classified as passes). The results have considerable potential value in providing guidance on using the TOEFL iBT Speaking scores for ITA screening.
Descriptors: Test Validity, Criterion Referenced Tests, English (Second Language), Language Tests, Second Language Learning, Scores, Standard Setting, Oral Language, Correlation, Test Use, Computer Assisted Testing, Teaching Assistants, Foreign Students, College Admission, Classification, Observation, Regression (Statistics), Predictor Variables, Cutting Scores, Higher Education, Screening Tests
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California (Los Angeles); Florida; North Carolina (Charlotte); Pennsylvania (Philadelphia)
Identifiers - Assessments and Surveys: Test of English as a Foreign Language
Grant or Contract Numbers: N/A