Mapping the speech code: Cortical responses linking the perception and production of vowels

The acoustic realization of speech is constrained by the physical mechanisms by which it is produced. Yet for speech perception, the degree to which listeners utilize experience derived from speech production has long been debated. In the present study, we examined how sensorimotor adaptation during production may affect perception, and how this relationship may be reflected in early vs. late electrophysiological responses. Participants first performed a baseline speech production task, followed by a vowel categorization task during which EEG responses were recorded. In a subsequent speech production task, half the participants received shifted auditory feedback, leading most to alter their articulations. This was followed by a second, post-training vowel categorization task. We compared changes in vowel production to both behavioral and electrophysiological changes in vowel perception. No differences in phonetic categorization were observed between groups receiving altered or unaltered feedback. However, exploratory analyses revealed correlations between vocal motor behavior and phonetic categorization. EEG analyses revealed correlations between vocal motor behavior and cortical responses in both early and late time windows. These results suggest that participants' recent production behavior influenced subsequent vowel perception. We suggest that the change in perception can be best characterized as a mapping of acoustics onto articulatio

La poursuite auditive du mouvement acoustique vers l'acquisition des catégories phonétiques

Cette thèse explore le développement de la parole chez l'enfant durant la première année de vie. Elle vise précisément à caractériser le mécanisme à la base de l'acquisition des catégories phonétiques. Les nombreuses recherches dans le domaine offrent un portrait compréhensible de la trajectoire développementale de la perception de la parole. Les mécanismes de ce développement demeurent toutefois mal compris, particulièrement en ce qui a trait aux stratégies de l'enfant pour faire face au problème de la variabilité. En s'inspirant des théories de l'invariance chez l'adulte et sur la base de données empiriques chez l'enfant, cette thèse présente trois études visant à soutenir l'hypothèse selon laquelle la poursuite auditive du mouvement acoustique sous-tend l'acquisition des catégories phonétiques. Tout d'abord, deux études de modélisation simulent l'acquisition d'un type particulier de catégories phonétiques, les tons lexicaux, par le biais de réseaux neuronaux artificiels de type non supervisé. Ces simulations évaluent l'impact de diverses sources de variabilité et l'efficacité du mouvement acoustique sur la catégorisation des tons en chinois mandarin. Les résultats montrent que malgré un degré modéré de variabilité, les patrons de fréquence fondamentale présentent des régularités permettant de distinguer les quatre tons mandarins, sans information préalable quant au nombre de catégories à découvrir. Ceci suggère que le signal acoustique continu peut suffire à l'acquisition des tons lexicaux, sans besoin de faire appel à un ensemble de propriétés phonétiques abstraites. En présence de multiples sources de variabilité cependant, l'information spectrale du signal de surface n'entretient qu'une faible relation avec les sons de la parole recherchés. À l'opposé, l'information dynamique (les profils de vélocité de la fréquence fondamentale) permet d'atteindre un niveau de performance comparable à celui de l'adulte pour l'identification des tons. De plus, les quatre profils de vélocité découverts par le réseau neuronal correspondent aux quatre tons mandarins et permettent de caractériser les gestes invariants impliqués dans la production tonale. Afin de vérifier si l'enfant peut faire usage de cette stratégie dynamique pour normaliser le signal de la parole, une étude comportementale examine ensuite la capacité d'enfants préverbaux à percevoir des variations acoustiques reflétant une contrainte articulatoire. Une procédure de regard préférentiel est utilisée afin de vérifier si des enfants de 4 et 8 mois peuvent distinguer entre eux des patrons d'intonation possibles et impossibles sur le plan articulatoire et produits par un locuteur inconnu. Les résultats montrent que les enfants des deux groupes d'âge écoutent plus longuement les stimuli possibles, indiquant qu'ils peuvent détecter des variations subtiles de vélocité de la fréquence fondamentale et préfèrent les variations qui respectent la contrainte articulatoire. Ces résultats suggèrent qu'en bas âge déjà, les enfants peuvent calculer la première dérivée d'informations spectrales continues et réduire la variabilité interlocuteur à partir de la dynamique du signal acoustique. Le modèle proposé par les études de simulations permet d'établir l'efficacité de l'information dynamique dans le développement phonétique. L'étude comportementale permet pour sa part de vérifier la sensibilité à cette information chez l'enfant en bas âge. Ces résultats suggèrent que l'invariance se situe à la fois au niveau acoustique/auditif, moteur et développemental, et que la poursuite auditive du mouvement acoustique reflétant les gestes articulatoires représente une stratégie efficace pour l'acquisition des catégories phonétiques. ______________________________________________________________________________ MOTS-CLÉS DE L’AUTEUR : Catégories phonétiques, Apprentissage distributionnel, Perception de la parole, Production de la parole, Acquisition du langage, Réseaux neuronaux artificiels non-supervisé

Auditory comprehension: from the voice up to the single word level

Auditory comprehension, the ability to understand spoken language, consists of a number of different auditory processing skills. In the five studies presented in this thesis I investigated both intact and impaired auditory comprehension at different levels: voice versus phoneme perception, as well as single word auditory comprehension in terms of phonemic and semantic content. In the first study, using sounds from different continua of ‘male’-/pæ/ to ‘female’-/tæ/ and ‘male’-/tæ/ to ‘female’-/pæ/, healthy participants (n=18) showed that phonemes are categorised faster than voice, in contradistinction with the common hypothesis that voice information is stripped away (or normalised) to access phonemic content. Furthermore, reverse correlation analysis suggests that gender and phoneme are processed on the basis of different perceptual representations. A follow-up study (same paradigm) in stroke patients (n=25, right or left hemispheric brain lesions, both with and without aphasia) showed that lesions of the right frontal cortex (likely ventral inferior frontal gyrus) leads to systematic voice perception deficits while left hemispheric lesions can elicit both voice and phoneme deficits. Together these results show that phoneme processing is lateralized while voice information processing requires both hemispheres. Furthermore, this suggests that commencing Speech and Language Therapy at a low level of acoustic processing/voice perception may be an appropriate method in the treatment of phoneme perception impairments. A longitudinal case study (CF) of crossed aphasia (rare acquired communication impairment secondary to lesion ipsilateral to the dominant hand) is then presented alongside a mini-review of the literature. Extensive clinical investigation showed that CF presented with word-finding difficulties related to impaired auditory phonological analysis, while functional Magnetic Resonance Imaging (fMRI) analyses showed right hemispheric lateralization of language functions (reading, repetition and verb generation). These results, together with the co-morbidity analysis from the mini-review, suggest that crossed aphasia can be explained by developmental disorders which cause partial right lateralization shift of language processes. Interestingly, in CF this process did not affect voice lateralization and information processing, suggesting partial segregation of voice and speech processing. In the last two studies, auditory comprehension was examined at the single word level using a word-picture matching task with congruent (correct target) and incongruent (semantic, phonological and unrelated foils) conditions. fMRI in healthy participants (n=16) revealed a key role of the pars triangularis (phonological processing), the left angular gyrus (semantic incongruency) and the left precuneus (semantic relatedness) in this task – regions typically associated via the arcuate fasciculus and often impaired in aphasia. Further investigation of stroke patients on the same task (n=15) suggested that the connections between the angular gyrus and the pars triangularis serve a fundamental role in semantic processing. The quality of a published word-picture matching task was also investigated, with results questioning the clinical relevance of this task as an assessment tool. Finally, a pilot study looking at the effect of a computer-assisted auditory comprehension therapy (React2©) in 6 stroke patients (vs. 6 healthy controls and 6 stroke patients without therapy) is presented. Results show that the more therapy patients carry out the more improvement is seen in the semantic processing of single nouns. However, these results need to be reproduced on a larger scale in order to generalise any outcomes. Overall, the findings from these studies present new insight into, as well as extending on, current cognitive and neuroanatomical models of voice perception, speech perception and single word auditory comprehension. A combinatorial approach to cognitive and neuroanatomical models is proposed in order to further research, and thus improve clinical care, into impaired auditory comprehension

Training the perception and production of English vowels /e/ and /æ/ by Cantonese-speaking secondary school students.

Wong, Wing Sze.Thesis (M.Phil.)--Chinese University of Hong Kong, 2010.Includes bibliographical references (leaves 215-241).Abstracts in English and Chinese; some appendixes include Chinese.Title Page --- p.iAcknowledgments --- p.iiAbstract --- p.ivTable of Contents --- p.viiiList of Tables --- p.xiiiList of Figures --- p.xviiChapter Chapter1 --- Introduction --- p.1Chapter Chapter2 --- Literature Review --- p.5Chapter 2.1 --- Phonology in Second Language Acquisition --- p.8Chapter 2.2 --- Modeling Speech Perception --- p.8Chapter 2.2.1 --- Speech Learning Model --- p.8Chapter 2.2.2 --- Perceptual Assimilation Model --- p.11Chapter 2.2.3 --- Native Language Magnet Model --- p.13Chapter 2.3 --- Linking Up Speech Perception and Production --- p.16Chapter 2.3.1 --- The Motor Theory --- p.17Chapter 2.3.2 --- The Direct Realist Approach to Speech Perception --- p.18Chapter 2.3.3 --- General Approach to Speech Perception --- p.20Chapter 2.4 --- Training in the Laboratory --- p.21Chapter 2.4.1 --- Discrimination vs. Identification Training --- p.22Chapter 2.4.2 --- High-Variability Phonetic Training (HVPT) --- p.25Chapter 2.4.3 --- Vowel Training Studies --- p.30Chapter 2.4.4 --- Perceptual Training on Production --- p.34Chapter 2.4.5 --- Summary of Previous Research --- p.38Chapter 2.5 --- Current Research Background --- p.38Chapter 2.5.1 --- Cantonese Vowel System vs. English Vowel System --- p.39Chapter 2.5.2 --- Cantonese Learners' Difficulties --- p.42Chapter 2.5.3 --- The Present Research --- p.44Chapter 2.6 --- Pilot Study --- p.45Chapter 2.6.1 --- Purpose --- p.45Chapter 2.6.2 --- Participants --- p.45Chapter 2.6.3 --- Procedures --- p.46Chapter 2.6.4 --- Results --- p.47Chapter 2.6.4.1 --- Perceptual Performance --- p.47Chapter 2.6.4.2 --- Production Performance --- p.49Chapter 2.6.5 --- Discussions & Suggestions --- p.50Chapter 2.7 --- Research Questions --- p.53Chapter Chapter3 --- Research Methodology --- p.54Chapter 3.1 --- Research Subjects --- p.54Chapter 3.2 --- Research Design and Procedures --- p.57Chapter 3.2.1 --- The Research Setting --- p.57Chapter 3.2.2 --- The Design in Details --- p.58Chapter 3.2.2.1 --- Before the Experiment: Preparation --- p.60Chapter 3.2.2.2 --- Phase 1: Pretest Phase --- p.61Chapter 3.2.2.2.1 --- Production Pretest: Word List Reading --- p.61Chapter 3.2.2.2.2 --- Perception Pretest: Identification test --- p.62Chapter 3.2.2.3 --- Phase 2: Training Phase --- p.65Chapter 3.2.2.3.1 --- The HVPT --- p.65Chapter 3.2.2.3.2 --- The LVPT --- p.67Chapter 3.2.2.3.3 --- Control --- p.67Chapter 3.2.2.4 --- Phase 3: Posttest Phase --- p.68Chapter 3.2.2.4.1 --- Production --- p.68Chapter 3.2.2.4.1.1 --- Production Post-test: Word List Reading --- p.68Chapter 3.2.2.4.1.2 --- Test of Contextualization (TC): Passage Reading --- p.68Chapter 3.2.2.4.2 --- Perception --- p.69Chapter 3.2.2.4.2.1 --- Perception Posttest: Identification Test --- p.69Chapter 3.2.2.4.2.1.1 --- Test of Generalization 1 (TG1): Identification Test --- p.69Chapter 3.2.2.4.2.1.2 --- Test of Generalization 2 (TG2): Identification Test --- p.69Chapter 3.2.3 --- Materials --- p.70Chapter 3.2.3.1 --- Stimuli --- p.70Chapter 3.2.3.2 --- Computer Training Programme --- p.73Chapter 3.2.3.3 --- Word Lists for Reading --- p.73Chapter 3.2.3.4 --- Technological Equipment --- p.74Chapter 3.2.3.5 --- Survey Forms --- p.74Chapter 3.2.4 --- Data Processing --- p.75Chapter 3.2.4.1 --- Data Transcription --- p.75Chapter 3.2.4.1.1 --- Procedures --- p.75Chapter 3.2.4.1.2 --- Reliability Checking --- p.76Chapter 3.2.4.1.2.1 --- Aim --- p.76Chapter 3.2.4.1.2.2 --- Intra-rater Reliability --- p.76Chapter 3.2.4.1.2.3 --- Inter-rater Reliability --- p.77Chapter 3.2.4.2 --- Data Scoring --- p.77Chapter 3.2.5 --- Data Analysis --- p.78Chapter Chapter4 --- Results --- p.79Chapter 4.1 --- Introduction --- p.79Chapter 4.2 --- Perceptual Performance --- p.80Chapter 4.2.1 --- Overall Performance --- p.80Chapter 4.2.2 --- Effects of the Training Approaches --- p.82Chapter 4.2.2.1 --- General Overview --- p.82Chapter 4.2.2.2 --- Statistical Analysis --- p.84Chapter 4.2.3 --- Effects of the Proficiency Level and Vowel Difference --- p.87Chapter 4.2.3.1 --- General Overview --- p.87Chapter 4.2.3.2 --- Statistical Analysis --- p.88Chapter 4.2.3.2.1 --- The HVPT Group --- p.88Chapter 4.2.3.2.2 --- The LVPT Group --- p.92Chapter 4.2.3.3 --- Summary --- p.94Chapter 4.2.4 --- Generalizability of the Training --- p.95Chapter 4.2.4.1 --- Test of Generalization 1 --- p.96Chapter 4.2.4.2 --- Test of Generalization 2 --- p.98Chapter 4.2.4.3 --- Summary --- p.100Chapter 4.2.5 --- Summing up the Results in Perceptual Identification Tests --- p.101Chapter 4.3 --- Production Performance --- p.102Chapter 4.3.1 --- Overall Performance --- p.102Chapter 4.3.2 --- Effects of the Training Approaches --- p.108Chapter 4.3.2.1 --- General Overview --- p.108Chapter 4.3.2.2 --- Statistical Analysis --- p.110Chapter 4.3.3 --- Effects of the Proficiency Level and Vowel Difference --- p.112Chapter 4.3.3.1 --- General Overview --- p.112Chapter 4.3.3.2 --- Statistical Analysis --- p.113Chapter 4.3.3.2.1 --- The HVPT Group..： --- p.113Chapter 4.3.3.2.2 --- The LVPT Group --- p.115Chapter 4.3.3.3 --- Summary --- p.119Chapter 4.3.4 --- Contextualizability of the Training --- p.120Chapter 4.3.5 --- Follow-up Acoustic Analysis for Production Posttest --- p.123Chapter 4.3.6 --- Summing up the Results in Production Tests --- p.130Chapter 4.4 --- Summary of the Chapter --- p.130Chapter Chapter5 --- Discussions --- p.132Chapter 5.1 --- Introduction --- p.132Chapter 5.2 --- Evaluation of the Training Approaches 一 Research Question 1 & 2 --- p.133Chapter 5.2.1 --- Review of Research Question 1 & 2 --- p.133Chapter 5.2.2. --- Perceptual Domain --- p.134Chapter 5.2.2.1 --- General Success of the HVPT and the LVPT Groupsin the Perceptual Learning --- p.135Chapter 5.2.2.1.1 --- Consistent Use of the Same Task --- p.135Chapter 5.2.2.1.2 --- Nature of the Training Tasks --- p.136Chapter 5.2.2.1.3 --- Use of Identification Tasks --- p.136Chapter 5.2.2.1.4 --- Adoption of Feedback --- p.137Chapter 5.2.2.2 --- Effectiveness of the HVPT over the LVPT --- p.138Chapter 5.2.2.2.1 --- Promotion of Selective Attention ´ؤ Supporting an Exemplar-based approach to Speech Perception --- p.139Chapter 5.2.2.2.2 --- Simulation of Real-life Experience --- p.141Chapter 5.2.3 --- Production Result --- p.142Chapter 5.2.3.1 --- The Relationship between Perception and Production --- p.143Chapter 5.2.3.1.1 --- The Motor Theory --- p.147Chapter 5.2.3.1.2 --- The Direct Realist Theory --- p.148Chapter 5.2.3.1.3 --- General Approach to Speech Perception --- p.149Chapter 5.2.4 --- Summary --- p.149Chapter 5.3 --- Generalizability of the Training Effects - Research Question 3 --- p.150Chapter 5.3.1 --- Review of Research Question 3 --- p.150Chapter 5.3.2. --- Perceptual Result --- p.151Chapter 5.3.2.1 --- Discussion of the Generalizability of the Training Effects --- p.152Chapter 5.3.3 --- Production Result --- p.156Chapter 5.3.3.1 --- Discussion of the results in Test of Contextualization --- p.157Chapter 5.3.4 --- Summary --- p.159Chapter 5.4 --- The Effect of Proficiency Groups - Research Question 4 --- p.160Chapter 5.4.1 --- Review of Research Question 4 --- p.160Chapter 5.4.2 --- Perceptual Aspect --- p.160Chapter 5.4.3 --- Production Aspect --- p.161Chapter 5.4.4 --- Discussion of the Effect of Proficiency Groups --- p.162Chapter 5.4.4.1 --- Perception and Production as a Process --- p.163Chapter 5.4.4.2 --- Indistinguishable Proficiency Levels of the Subjects --- p.165Chapter 5.4.5 --- Summary --- p.166Chapter 5.5 --- The Effect of Vowels - Research Question 5 --- p.166Chapter 5.5.1 --- Review of Research Question 5 --- p.166Chapter 5.5.2 --- Perceptual Aspect --- p.167Chapter 5.5.2.1 --- Discussion of the Effect of Vowels on Perceptual Learning --- p.168Chapter 5.5.2.2 --- Speech Learning Model --- p.169Chapter 5.5.2.3 --- Perceptual Assimilation Model --- p.170Chapter 5.5.3 --- Production Aspect --- p.172Chapter 5.5.3.1 --- Discussion of the Effect of Vowel on Production --- p.173Chapter 5.5.4 --- Summary --- p.175Chapter 5.6 --- Summary of the Chapter --- p.176Chapter Chapter6 --- Conclusion --- p.178Chapter 6.1 --- An Overview of the Current Study --- p.178Chapter 6.2 --- Contributions of the Current Study --- p.180Chapter 6.3 --- Limitations of the Current Study --- p.183Chapter 6.4 --- Future Research Directions and Implications --- p.185AppendicesAppendix A Consent Form & Survey Form --- p.188Appendix B Language Background of the Subjects --- p.191Appendix C Production Word List & Reading Passage --- p.193Appendix D Perceptual Training Tokens --- p.196Appendix E Perceptual Performance in all Tests --- p.199Appendix F Perceptual Performance in the Pretest and the Posttest --- p.201Appendix G Perceptual Performance in two Tests of Generalization --- p.203Appendix H Production Performance in all Tests --- p.207Appendix I Production Performance in the Pretest and the Posttest --- p.209Appendix J Production Performance in Test of Contextualization --- p.211Appendix K Perception and Production Performance in contrast --- p.213References --- p.21

Language Science Meets Cognitive Science: Categorization and Adaptation

Questions of domain-generality—the extent to which multiple cognitive functions are represented and processed in the same manner—are common topics of discussion in cognitive science, particularly within the realm of language. In the present dissertation, I examine the domain-specificity of two processes in speech perception: category learning and rate adaptation. With regard to category learning, I probed the acquisition of categories of German fricatives by English and German native speakers, finding a bias in both groups towards quicker acquisition of non-disjunctive categories than their disjunctive counterparts. However, a study using an analogous continuum of non-speech sounds, in this case spectrally-rotated musical instrument sounds, did not show such a bias, suggesting that at least some attributes of the phonetic category learning process are unique to speech. For rate adaptation, meanwhile, I first report a study examining rate adaptation in Modern Standard Arabic (MSA), where consonant length is a contrastive part of the phonology; that is, where words can be distinguished from one another by the length of the consonants that make them up. I found that changing the rate of the beginning of a sentence can lead a consonant towards the end of the sentence to change in its perceived duration; a short consonant can sound like a long one, and a long consonant can sound like a short one. An analogous experiment examined rate adaptation in event segmentation, where adaptation-like effects had not previously been explored, using recordings of an actor interacting with a touchscreen. I found that the perception of actions can also be affected by the rate of previously-occurring actions. Listeners adapt to the rate at the beginning of a series of actions when deciding what they saw last in that series of actions. This suggests that rate adaptation follows similar lines across both domains. All told, this dissertation leads to a picture of domain-specificity in which both domain-general and domain-specific processes can operate, with domain-specific processes can help scaffold the use of domain-general processing