Article thumbnail

Speech Cues Contribute to Audiovisual Spatial Integration

By Christopher W. Bishop and Lee M. Miller

Abstract

Speech is the most important form of human communication but ambient sounds and competing talkers often degrade its acoustics. Fortunately the brain can use visual information, especially its highly precise spatial information, to improve speech comprehension in noisy environments. Previous studies have demonstrated that audiovisual integration depends strongly on spatiotemporal factors. However, some integrative phenomena such as McGurk interference persist even with gross spatial disparities, suggesting that spatial alignment is not necessary for robust integration of audiovisual place-of-articulation cues. It is therefore unclear how speech-cues interact with audiovisual spatial integration mechanisms. Here, we combine two well established psychophysical phenomena, the McGurk effect and the ventriloquist's illusion, to explore this dependency. Our results demonstrate that conflicting spatial cues may not interfere with audiovisual integration of speech, but conflicting speech-cues can impede integration in space. This suggests a direct but asymmetrical influence between ventral ‘what’ and dorsal ‘where’ pathways

Topics: Research Article
Publisher: Public Library of Science
OAI identifier: oai:pubmedcentral.nih.gov:3166076
Provided by: PubMed Central

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.

Suggested articles

Citations

  1. (2009). A Multisensory Cortical Network for Understanding Speech in Noise.
  2. (2007). Attention to touch weakens audiovisual speech integration.
  3. (2005). Audiovisual integration of speech falters under high attention demands.
  4. (2003). Auditory influences on visual temporal rate perception.
  5. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors.
  6. (1999). Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex.
  7. (1994). Exploring the relation between McGurk Interference and Ventriloquism.
  8. (2003). Gaze behavior in audiovisual speech perception: the influence of ocular fixations on the McGurk effect.
  9. (2000). Hearing by eye: how much spatial degradation can be tolerated?
  10. (1976). Hearing lips and seeing voices.
  11. (1966). Human Spatial Orientation.
  12. (2002). Humans integrate visual and haptic information in a statistically optimal fashion.
  13. (1983). Interactions among converging sensory inputs in the superior colliculus.
  14. (2008). Look who’s talking: the deployment of visuo-spatial attention during multisensory speech processing under noisy environmental conditions.
  15. (2004). Merging the senses into a robust percept.
  16. (2009). Multisensory integration enhances phonemic restoration.
  17. (2006). Multisensory integration of speech signals: the relationship between space and time.
  18. (1983). Object vision and spatial vision: two cortical pathways.
  19. (2007). Physiological and anatomical evidence for multisensory interactions in auditory cortex.
  20. (2001). Rules of intersensory integration in spatial scene analysis and speechreading.
  21. (1953). Some Epxeriments on the Recognition of Speech, with One and with Two Ears.
  22. (1996). Spatial determinants of multisensory integration in cat superior colliculus neurons.
  23. (1996). Spatial Hearing - Revised Edition: The Psychophysics of Human Sound Localization.
  24. (2001). Temporal and spatial dependency of the ventriloquism effect.
  25. (1997). The effects of separating auditory and visual sources on audovisual integration of speech.
  26. (1981). The role of visual-auditory "compellingness" in the ventriloquism effect: implications for transitivity among the spatial senses.
  27. (2004). The ventriloquist effect results from near-optimal bimodal integration.
  28. (1954). Visual Contribution to Speech Intelligibility in Noise.
  29. (2009). Visual influences on ferret auditory cortex.
  30. (2008). Visual-auditory spatial processing in auditory cortical neurons.
  31. (1992). Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection.
  32. (2000). What’, ‘where’ and ‘how’ in auditory cortex.