ERIC Number: EJ1104405
Record Type: Journal
Publication Date: 2016-Jul
Abstractor: As Provided
Construct Validity in TOEFL iBT Speaking Tasks: Insights from Natural Language Processing
Kyle, Kristopher; Crossley, Scott A.; McNamara, Danielle S.
Language Testing, v33 n3 p319-340 Jul 2016
This study explores the construct validity of speaking tasks included in the TOEFL iBT (e.g., integrated and independent speaking tasks). Specifically, advanced natural language processing (NLP) tools, MANOVA difference statistics, and discriminant function analyses (DFA) are used to assess the degree to which and in what ways responses to these tasks differ with regard to linguistic characteristics. The findings lend support to using a variety of speaking tasks to assess speaking proficiency. Namely, with regard to linguistic differences, the findings suggest that responses to performance tasks can be accurately grouped based on whether a task is independent or integrated. The findings also suggest that although the independent tasks included in the TOEFL iBT may represent a single construct, responses to integrated tasks vary across task sub-type.
Descriptors: Construct Validity, Natural Language Processing, Speech Skills, Speech Acts, Task Analysis, English (Second Language), Language Tests, Lexicology, Statistical Analysis, Integrated Activities, Effect Size, Preferences, College Entrance Examinations, English Curriculum, Second Language Learning, Multivariate Analysis
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: email@example.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language