Analyse de scènes de parole multisensorielle : Mise en évidence et caractérisation d’un processus de liage audiovisuel préalable à la fusion.

This thesis is focused on audiovisual speech, with the aim of better understanding the mechanisms enabling to link auditory and visual speech signals in the human brain.In a first theoretical part were view key issues concerning audiovisual speech and the underlying models of integration of auditory and visual inputs. Then we discuss the question of the existence of an early audio-visual integration stage and we formulate our main hypothesis in which we posit the existence of an “audiovisual binding process”, prior to audio- visual fusion.Based on this assumption the experimental work is organized in two parts. Firstly we provide evidence in favor of the existence of a binding process by developing and gradually refining a new experimental paradigm. Then we attempt to characterize this process by setting up a number of experiments studying the influence of various critical parameters. We finish with a discussion of our results and relate them to present questions about cognitive perceptual processing of audiovisual speech.Dans le cadre de cette thèse centrée sur la perception de la parole audiovisuelle, le but de notre étude est de mieux comprendre les mécanismes qui permettent de lier les signaux auditifs et visuels de la parole. Une première partie théorique permet dépasser en revue les principales questions portant sur la parole audiovisuelle et sur les principaux modèles d’intégration des inputs auditif et visuels. Puis nous abordons la question de l’existence d’un possible niveau précoce d’intégration audio-visuelle qui nous conduit à formuler notre hypothèse principale d’existence d’un processus de« liage audiovisuel », qui constituerait une étape préalable à la fusion audio-visuelle. En nous basant sur cette hypothèse nous abordons la phase expérimentale de notre travail, organisée en deux parties. D’abord nous entreprenons de démontrer l’existence du processus du liage en utilisant et en affinant peu à peu un nouveau paradigme expérimental. Puis nous traitons de la caractérisation de ce processus en mettant en place des expériences qui étudient l’influence de différents paramètres critiques. Nous terminons ce manuscrit par une discussion sur nos résultats et leur positionnement dans la littérature actuelle

Audiovisual binding in speech perception

International audienceWe have been elaborating in the last years in Grenoble a series of experimental works in which we attempt to show that audiovisual speech perception comprises an “audiovisual binding” stage before fusion and decision. This stage would be in charge to extract and associate the auditory and visual cues corresponding to a given speech source, before further categorisation processes could take place at a higher stage. We developed paradigms to characterize audiovisual binding in terms of both “streaming” and “chunking” adequate pieces of information. This can lead to elements of a possible computational model, in relation with a larger theoretical perceptuo-motor framework for speech perception, the “Perception-for-Action-Control” Theory

Audiovisual binding in speech perception

International audienceWe have been elaborating in the last years in Grenoble a series of experimental works in which we attempt to show that audiovisual speech perception comprises an “audiovisual binding” stage before fusion and decision. This stage would be in charge to extract and associate the auditory and visual cues corresponding to a given speech source, before further categorisation processes could take place at a higher stage. We developed paradigms to characterize audiovisual binding in terms of both “streaming” and “chunking” adequate pieces of information. This can lead to elements of a possible computational model, in relation with a larger theoretical perceptuo-motor framework for speech perception, the “Perception-for-Action-Control” Theory