ERIC Number: EJ1090046
Record Type: Journal
Publication Date: 2016-Feb
Abstractor: As Provided
Reference Count: 30
The Effects of Item Preview on Video-Based Multiple-Choice Listening Assessments
Koyama, Dennis; Sun, Angela; Ockey, Gary J.
Language Learning & Technology, v20 n1 p148-165 Feb 2016
Multiple-choice formats remain a popular design for assessing listening comprehension, yet no consensus has been reached on how multiple-choice formats should be employed. Some researchers argue that test takers must be provided with a preview of the items prior to the input (Buck, 1995; Sherman, 1997); others argue that a preview may decrease the authenticity of the task by changing the way input is processed (Hughes, 2003). Using stratified random sampling techniques, more and less proficient Japanese university English learners (N = 206) were assigned one of three test conditions: preview of question stem and answer options (n = 67), preview of question stem only (n = 70), and no preview (n = 69). A two-way ANOVA, with test condition and listening proficiency level as independent variables and score on the multiple-choice listening test as the dependent variable, indicated that the amount of item preview affected test scores but did not affect high and low proficiency students' scores differently. Item-level analysis identified items that were harder or easier than expected for one or more of the conditions, and the researchers posit three possible sources for these unexpected findings: 1) frequency of options in the input, 2) location of item focus, and 3) presence of organizational markers.
Descriptors: Multiple Choice Tests, Listening Comprehension Tests, Statistical Analysis, Language Proficiency, Japanese, Native Language, College Students, English (Second Language), Second Language Learning, Second Language Instruction, Foreign Countries, Test Format, Test Items, Sampling, Scores, Item Analysis, Asians, Linguistic Input, Language Processing, Video Technology, Teaching Methods
University of Hawaii National Foreign Language Resource Center. 1859 East-West Road #106, Honolulu, HI 96822. Tel: 808-956-9424; Fax: 808-956-5983; e-mail: firstname.lastname@example.org; Web site: http://llt.msu.edu
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Japan