NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ934651
Record Type: Journal
Publication Date: 2011-Aug
Pages: 17
Abstractor: As Provided
Reference Count: 66
ISBN: N/A
ISSN: ISSN-0096-1523
Psychophysics of the McGurk and Other Audiovisual Speech Integration Effects
Jiang, Jintao; Bernstein, Lynne E.
Journal of Experimental Psychology: Human Perception and Performance, v37 n4 p1193-1209 Aug 2011
When the auditory and visual components of spoken audiovisual nonsense syllables are mismatched, perceivers produce four different types of perceptual responses, auditory correct, visual correct, fusion (the so-called "McGurk effect"), and combination (i.e., two consonants are reported). Here, quantitative measures were developed to account for the distribution of the four types of perceptual responses to 384 different stimuli from four talkers. The measures included mutual information, correlations, and acoustic measures, all representing audiovisual stimulus relationships. In Experiment 1, open-set perceptual responses were obtained for acoustic /b[script a]/ or /l[script a]/ dubbed to video /b[script a], d[script a], g[script a], v[script a], z[script a], l[script a], w[script a], [eth][script a]/. The talker, the video syllable, and the acoustic syllable significantly influenced the type of response. In Experiment 2, the best predictors of response category proportions were a subset of the physical stimulus measures, with the variance accounted for in the perceptual response category proportions between 17% and 52%. That audiovisual stimulus relationships can account for perceptual response distributions supports the possibility that internal representations are based on modality-specific stimulus relationships. (Contains 1 footnote, 6 tables, and 6 figures.)
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002-4242. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: order@apa.org; Web site: http://www.apa.org/publications
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A