Multimodal perception of speech segments and speech prosody in relationship with their production

Abstract

International audienceThe aim of this talk will be to investigate multimodal perception of speech linked to its production. Both segmental and suprasegmental aspects of speech will be considered. We shall first recall how speech sounds and images are fused in speech perception, starting from speech in noise and the McGurk effect. This will show that speech communication is intrinsically multimodal. We shall discuss possible cognitive architectures for audiovisual fusion. These will be related to recent neurocognitive data on perceptuo-motor links in the human brain, from mirror neurons to the cortical dorsal route of speech perception. Then we shall present a number of recent data we have obtained on audiovisual prosody, dealing with the audiovisual perception and production of contrastive focus