NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1015545
Record Type: Journal
Publication Date: 2013-Apr
Pages: 10
Abstractor: As Provided
ISSN: ISSN-1092-4388
Gaze Patterns and Audiovisual Speech Enhancement
Yi, Astrid; Wong, Willy; Eizenman, Moshe
Journal of Speech, Language, and Hearing Research, v56 n2 p471-480 Apr 2013
Purpose: In this study, the authors sought to quantify the relationships between speech intelligibility (perception) and gaze patterns under different auditory-visual conditions. Method: Eleven subjects listened to low-context sentences spoken by a single talker while viewing the face of one or more talkers on a computer display. Subjects either maintained their gaze at a specific distance (0 degrees, 2.5 degrees, 5 degrees, 10 degrees, and 15 degrees) from the center of the talker's mouth (CTM) or moved their eyes freely on the computer display. Eye movements were monitored with an eye-tracking system, and speech intelligibility was evaluated by the mean percentage of correctly perceived. Results: With a single talker and a fixed point of gaze, speech intelligibility was similar for all fixations within 10 degrees of the CTM. With visual cues from two talker faces and a speech signal from one of the talkers, speech intelligibility was similar to that of a single talker for fixations within 2.5 degrees of the CTM. With natural viewing of a single talker, gaze strategy changed with speech-signal-to-noise ratio (SNR). For low speech-SNR, a strategy that brought the point of gaze directly to within 2.5 degrees of the CTM was used in approximately 80% of trials, whereas in high speech-SNR it was used in only approximately 50% of trials. Conclusions: With natural viewing of a single talker and high speech-SNR, subjects can shift their gaze between points on the talker's face without compromising speech intelligibility. With low-speech SNR, subjects change their gaze patterns to fixate primarily on points that are in close proximity to the talker's mouth. The latter strategy is essential to optimize speech intelligibility in situations where there are simultaneous visual cues from multiple talkers (i.e., when some of the visual cues are distracters). (Contains 7 figures and 1 footnote.)
American Speech-Language-Hearing Association (ASHA). 10801 Rockville Pike, Rockville, MD 20852. Tel: 800-638-8255; Fax: 301-571-0457; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A