Data and simulations about audiovisual asynchrony and predictability in speech perception

International audienceSince a paper by Chandrasekaran et al. (2009), an increasing number of neuroscience papers capitalize on the assumption that visual speech would be typically 150 ms ahead of auditory speech. It happens that the estimation of audiovisual asynchrony by Chandrasekaran et al. is valid only in very specific cases, for isolated CV syllables or at the beginning of a speech utterance. We present simple audiovisual data on plosive-vowel syllables (pa, ta, ka, ba, da, ga, ma, na) showing that audiovisual synchrony is actually rather precise when syllables are chained in sequences, as they are typically in most parts of a natural speech utterance. Then we discuss on the way the natural coordination between sound and image (combining cases of lead and lag of the visual input) is reflected in the so-called temporal integration window for audiovisual speech perception (van Wassenhove et al., 2007). We conclude by a computational proposal about predictive coding in such sequences, showing that the visual input may actually provide and enhance predictions even if it is quite synchronous with the auditory input

Speech production after glossectomy: methodological aspects

International audienceThis article focuses on methodological issues related to quantitative assessments of speech quality after glossectomy. Acoustic and articulatory data were collected for 8 consonants from two patients. The acoustic analysis is based on spectral moments and the Klatt VOT. Lingual movements are recorded with ultrasound without calibration. The variations of acoustic and articulatory parameters across pre- and post-surgery conditions are analyzed in the light of perceptual evaluations of the stimuli. A parameter is considered to be relevant if its variation is congruent with perceptual ratings. The most relevant acoustic parameters are the skewness and the Center of Gravity. The Klatt VOT explains differences that could not be explained by spectral parameters. The SNTS ultrasound parameter provides information to describe impairments not accounted for by acoustical parameters. These results suggest that the combination of articulatory, perceptual and acoustic data provides comprehensive complementary information for a quantitative assessment of speech after glossectomy

Interaction between articulatory gestures and inner speech in a counting task

International audienceInteraction between covert and overt orofacial gestures has been poorly studied apart from old and rather qualitative experiments. The question deserves special interest in the context of the debate between auditory and motor theories of speech perception, where dual tasks may be of great interest. It is shown here that dynamic mandible and lips movement produced by a participant result in strong and stable perturbations to an inner speech counting task that has to be realized at the same time, while static orofacial configurations and static or dynamic manual actions produce no perturbation. This enables the authors to discuss how such kinds of orofacial perturbations could be introduced in dual task paradigms to assess the role of motor processes in speech perception

De bé à bébé : le transfert d'apprentissage auditori-moteur pour interroger l'unité de production de la parole

International audienceSpeech is often described as a sequence of units associating linguistic, sensory and motor representations. Are these representations linked at the level of a specific unit, for example, the syllable or the word? In the present study, we contrast these two hypotheses. We modified the production of the syllable “bé” (/be/) in French speakers using an auditory-motor adaptation paradigm that consists in altering the speakers’ auditory feedback. We studied how this modification then transfers to the production of the word “bébé” (/bebe/). The results suggest a link between linguistic and motor representations both at the word and the syllable level. They also show an effect of the position of the syllable in the transfer word, which raises new interrogations about serialcontrol of speech.La parole est souvent décrite comme une mise en séquence d'unités associant des représentations linguistiques, sensorielles et motrices. Le lien entre ces représentations se fait-il de manière privilégiée sur une unité spécifique ? Par exemple, est-ce la syllabe ou le mot ? Dans cette étude, nous voulons contraster ces deux hypothèses. Pour cela, nous avons modifié chez des locuteurs du français la production de la syllabe « bé », selon un paradigme d'adaptation auditori-motrice, consistant à perturber le retour auditif. Nous avons étudié comment cette modification se transfère ensuite à la production du mot « bébé ». Les résultats suggèrent un lien entre représentations linguistiques et motrices à plusieurs niveaux, à la fois celui du mot et de la syllabe. Ils montrent également une influence de la position de la syllabe dans le mot sur le transfert, qui soulève de nouvelles questions sur le contrôle sériel de la parole

A study of lip movements during spontaneous dialog and its application to voice activity detection

International audienceThis paper presents a quantitative and comprehensive study of the lip movements of a given speaker in different speech/nonspeech contexts, with a particular focus on silences i.e., when no sound is produced by the speaker . The aim is to characterize the relationship between "lip activity" and "speech activity" and then to use visual speech information as a voice activity detector VAD . To this aim, an original audiovisual corpus was recorded with two speakers involved in a face-to-face spontaneous dialog, although being in separate rooms. Each speaker communicated with the other using a microphone, a camera, a screen, and headphones. This system was used to capture separate audio stimuli for each speaker and to synchronously monitor the speaker's lip movements. A comprehensive analysis was carried out on the lip shapes and lip movements in either silence or nonsilence i.e., speech+nonspeech audible events . A single visual parameter, defined to characterize the lip movements, was shown to be efficient for the detection of silence sections. This results in a visual VAD that can be used in any kind of environment noise, including intricate and highly nonstationary noises, e.g., multiple and/or moving noise sources or competing speech signals

From Sensorimotor Experience To Speech Unit -Adaptation to altered auditory feedback in speech to assess transfer of learning in complex serial movements

International audienceUsing bird song as a model to understand generalization in motor learning, Hoffman and Sober recently found that adaptation to pitch-shift of birds’ vocal output transfered to the production of the same sounds embedded in a different serial context (J. Neurosc 2014). In humans, speech learning has been found to transfer as a function of the acoustical similarity between the training and the testing utterances (Cai et al. 2010, Rochet-Capellan et al. 2011) but it is unclear if transfer of learning is sensitive to serial order. We investigate the effects of serial order on transfer of speech motor learning using non-words sequences of CV syllables. Three groups of native speakers of French were trained to produce the syllable /be/ repetitively while their auditory feedback was altered in real time toward /ba/. They were then tested for transfer toward /be/ (control), /bepe/ or /pebe/ under normal feedback conditions. The training utterance was then produced again to test for after-effects. The auditory shift was achieved in real time using Audapter software (Cai et al. 2008). Adaptation and transfer effects were quantified in terms of changes in formants frequencies of the vowel /e/, as a function of its position and the preceding consonant in the utterance. Changes in formant frequencies in a direction opposite to the shift were significant for ~80% of the participants. Adaptation was still significant for the three groups in the after-effect block. Transfer effects in the /bepe/ and /pebe/ groups were globally smaller than that of the control group, particularly when the vowel /e/ came after /p/ and/or was in second position in the utterance. Taken together, the results suggest that transfer of speech motor learning is not homogenous and as observed by Hoffman and Sober, depends on the serial context of a sound within the utterance.Cai S, Boucek M, Ghosh SS, Guenther FH, Perkell JS. (2008). A system foronline dynamic perturbation of formant frequencies and results from perturbation of the Mandarin triphthong /iau/. In Proceedings of the 8th Intl. Seminar on Speech Production, Strasbourg, France, Dec. 8-12, 2008. pp. 65Cai, S., Ghosh, S. S., Guenther, F. H., & Perkell, J. S. (2010). Adaptive auditory feedback control of the production of formant trajectories in the Mandarin triphthong/iau/and its pattern of generalization. The Journal of the Acoustical Society of America, 128(4), 2033-2048.Hoffmann, L. A., & Sober, S. J. (2014). Vocal generalization depends on gesture identity and sequence. The Journal of Neuroscience, 34(16), 5564-5574.Rochet-Capellan, A., Richer, L., & Ostry, D. J. (2012). Nonhomogeneous transfer reveals specificity in speech motor learning. Journal of neurophysiology, 107(6), 1711-1717

Réorganisation du conduit vocal et réorganisation corticale de la parole : de la perturbation aux lèvres à la glossectomie. Études acoustiques et IRMf

21 pagesRetrouver l'usage de la langue - articulateur central de la parole - pour produire les dix voyelles du français, et ce après une opération de glossectomie suivie d'une reconstruction linguale à base d'un muscle de la cuisse (le gracilis), tel était le difficile problème que le patient K.H. (53 ans) a su résoudre au bout de neuf mois. Comment a-t-il pu apprendre à contrôler sa nouvelle " langue" pour produire les différentes voyelles de façon distincte ? C'est ce que nous avons cherché à comprendre dans cette étude. Nous avons mis en place un suivi expérimental longitudinal du patient en enregistrant ses productions acoustiques et son activité cérébrale juste avant l'opération, un mois après l'opération et neuf mois après l'opération. Nous avons ainsi pu suivre la récupération corticale du sujet, en l'occurrence la re-latéralisation dans l'hémisphère gauche pour l'articulation des catégories vocaliques de sa langue, en relation avec l'amélioration de ses performances articulatoires et acoustiques. Plus fondamentalement, cette prouesse atteste de la plasticité corticale phonologique via l'équifinalité compensatoire du système de la langue, en l'occurrence par le jeu des équivalence motrices et acoustiques

Evolution des activations cérébrales lors de la production de parole après exérèse au niveau de la cavité orale

International audienceUsing functional MRI, acoustic data and motor oral assessment, brain correlates of speech recovery after dramatic structural changes in the vocal tract have been investigated during three tasks: orofacial movements and speech production (vowels and syllables). Eleven patients were recorded during three sessions, preoperatively and postoperatively, 1 month and 3 months after surgery (for seven of them also 9 months after surgery). Eleven healthy subjects were recorded in parallel. A "whole brain" group analysis (patients/healthy subjects) reveals a specific activation pattern for patients, in particular during vowel task, which is session-dependent. Moreover, global lower brain activation compared to healthy subjects was found for all tasks in primary, secondary and integrative sensorimotor regions. Differences are interpreted in relation with the emergence of new speech motor goals and changes in the internal models.A partir de données d'IRM fonctionnelle, acoustiques et praxiques, nous avons étudié les corrélats cérébraux de l'adaptation de la parole après une modification structurelle importante du conduit vocal lors de trois tâches : mouvements oro-faciaux silencieux, voyelles et syllabes. Onze patients ont été enregistrés lors de trois sessions, pré-opératoire et post-opératoire à 1 mois et 3 mois (pour sept d'entre eux aussi à 9 mois). Onze sujets contrôles ont été enregistrés en parallèle. Une analyse de groupe (patients/contrôles) " cerveau entier " révèle des patrons d'activation spécifiques aux patients au cours des différentes sessions en particulier pour la tâche de production de voyelles. De plus, une moindre activation cérébrale par rapport aux sujets sains a été observée pour toutes les tâches dans des régions motrices, sensorielles et d'intégration sensori-motrice. Nous interprétons ces résultats en relation avec la redéfinition des buts de parole et l'adaptation de modèles internes du système moteur périphérique

Singing voice research at the Speech and Cognition Department, GIPSA-lab, Grenoble, France

International audienceThe Speech and Cognition Department of GIPSA-lab is a leading site for singing-voice research in France, linked to several laboratories in France and abroad. Several experimental platforms have been developed at the Speech and Cognition Department of GIPSA-lab to study voice and speech, communication behaviors in the case of human-human or computer-human interaction, and to evaluate physical models of human voice production. Selected studies conducted on the singing voice at the Speech and Cognition Department of GIPSA-lab will now be briefly described. All these studies result from national and/or international collaborations