Cortical mechanisms of seeing and hearing speech

In face-to-face communication speech is perceived through eyes and ears. The talker's articulatory gestures are seen and the speech sounds are heard simultaneously. Whilst acoustic speech can be often understood without visual information, viewing articulatory gestures aids hearing substantially in noisy conditions. On the other hand, speech can be understood, to some extent, by solely viewing articulatory gestures (i.e., by speechreading). In this thesis, electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) were utilized to disclose cortical mechanisms of seeing and hearing speech. One of the major challenges of modern cognitive neuroscience is to find out how the brain integrates inputs from different senses. In this thesis, integration of seen and heard speech was investigated using EEG and MEG. Multisensory interactions were found in the sensory-specific cortices at early latencies and in the multisensory regions at late latencies. Viewing other person's actions activate regions belonging to the human mirror neuron system (MNS) which are also activated when subjects themselves perform actions. Possibly, the human MNS enables simulation of other person's actions, which might be important also for speech recognition. In this thesis, it was demonstrated with MEG that seeing speech modulates activity in the mouth region of the primary somatosensory cortex (SI), suggesting that also the SI cortex is involved in simulation of other person's articulatory gestures during speechreading. The question whether there are speech-specific mechanisms in the human brain has been under scientific debate for decades. In this thesis, evidence for the speech-specific neural substrate in the left posterior superior temporal sulcus (STS) was obtained using fMRI. Activity in this region was found to be greater when subjects heard acoustic sine wave speech stimuli as speech than when they heard the same stimuli as non-speech.reviewe

Emerging Linguistic Functions in Early Infancy

This paper presents results from experimental studies on early language acquisition in infants and attempts to interpret the experimental results within the framework of the Ecological Theory of Language Acquisition (ETLA) recently proposed by (Lacerda et al., 2004a). From this perspective, the infant’s first steps in the acquisition of the ambient language are seen as a consequence of the infant’s general capacity to represent sensory input and the infant’s interaction with other actors in its immediate ecological environment. On the basis of available experimental evidence, it will be argued that ETLA offers a productive alternative to traditional descriptive views of the language acquisition process by presenting an operative model of how early linguistic function may emerge through interaction

Maturing Temporal Bones as Non-Neural Sites for Transforming the Speech Signal during Language Development

Developmental events in the temporal bones shift the pattern of a given speech sounds acoustic profile through the time children are mapping linguistic sound systems. Before age 5 years, frequency information in vowels is differentially accessible through the years children are acquiring the sound systems of their native language(s). To model the acoustic effects caused by developing temporal bones, data collected to elicit steady-state vowels from adult native speakers of English and Diné were modified to reflect the form of children\u27s hearing sensitivities at different ages based on patterns established in the psychoacoustic literature. It was assumed, based on the work of psychacousticians (e.g., Werner, Fay & Popper 2012; and Werner & Marean 1996), that the effects caused by immature temporal bones were conductive immaturities, and the age-sensitive filters were constructed based on psychoacoustic research into the hearing of infants and children. Data were partitioned by language, sex, and individual vowels and compared for points of similarity and difference in the way information in vowels is filtered because of the constraints imposed by the immaturity of the temporal bones. Results show that the early formant pattern becomes successively modified in a constrained pattern reflecting maturational processes. Results also suggest that children may well be switching strategies for processing vowels, using a more adult-like process after 18 months. Future research should explore if early hearing not only affects individual speech sounds but their relationships to one another in the vowel space as well. Additionally, there is an interesting artifact in the observed gradual progression to full adult hearing which may be the effect of the foramen of Huschke contributing to the filters at 1 year and 18 months. Given that immature temporal bones reflect brain expansion and rotational birth in hominids, these results contribute to the discussion of the biological underpinnings of the evolution of language.\u2

The Relationship of Somatosensory Perception and Fine-Force Control in the Adult Human Orofacial System

The orofacial area stands apart from other body systems in that it possesses a unique performance anatomy whereby oral musculature inserts directly into the underlying cutaneous skin, allowing for the generation of complex three-dimensional deformations of the orofacial system. This anatomical substrate provides for the tight temporal synchrony between self-generated cutaneous somatosensation and oromotor control during functional behaviors in this region and provides the necessary feedback needed to learn and maintain skilled orofacial behaviors. The Directions into Velocity of Articulators (DIVA) model highlights the importance of the bidirectional relationship between sensation and production in the orofacial region in children learning speech. This relationship has not been as well-established in the adult orofacial system. The purpose of this observational study was to begin assessing the perception-action relationship in healthy adults and to describe how this relationship may be altered as a function of healthy aging. This study was designed to determine the correspondence between orofacial cutaneous perception using vibrotactile detection thresholds (VDT) and low-level static and dynamic force control tasks in three representative age cohorts. Correlational relationships among measures of somatosensory capacity and low-level skilled orofacial force control were determined for 60 adults (19-84 years). Significant correlational relationships were identified using non-parametric Spearman’s correlations with an alpha at 0.1 between the 5 Hz test probe and several 0.5 N low-level force control assessments in the static and slow ramp-and-hold condition. These findings indicate that as vibrotactile detection thresholds increase (labial sensation decreases), ability to maintain a low-level force endpoint decreases. Group data was analyzed using non-parametric Kruskal-Wallis tests and identified significant differences between the 5 Hz test frequency probe and various 0.5 N skilled force assessments for group variables such as age, pure tone hearing assessments, sex, speech usage and smoking history. Future studies will begin the processing of modeling this complex multivariate relationship in healthy individuals before moving to a disordered population

Role of the speech motor system in speech perception

Puheen havaitsemisella viitataan prosessiin, jonka puitteissa kielen äänteet kuullaan, tulkitaan ja ymmärretään. Vaikka tämä prosessi saattaa kuulostaa mitäänsanomattoman yksinkertaiselta, ovat ne neuraaliset mekanismit, jotka sen mahdollistavat, kaikkea muuta kuin yksinkertaisia. Keskeinen tutkijoiden päitä vaivaava kysymys puheen havaitsemisessa onkin, miten kuulija poimii olennaisen informaation puhesignaalista? Vastauksen etsintä on synnyttänyt toisistaan selvästi erillään olevia teoreettisia näkökulmia, joiden keskeisin erottava tekijä voidaan esittää kysymyksen muodossa: mikä on puheen motorisen järjestelmän rooli puheen havaitsemisessa? Vai onko sillä roolia laisinkaan? Toisin sanoen, onko niillä aivoalueilla ja -rakenteilla, jotka vastaavat puheen tuottamisesta, osuutta myös puheen havaitsemisessa? Tässä työssä puheen motorisen järjestelmän roolia puheen havaitsemisessa tutkittiin käyttäen tutkimusmenetelmänä magnetoenkefalografiaa (MEG). Puheen havaitsemisen perustana olevia neuraalisia mekanismeja tutkittiin neljässä erilaisessa havaitsemistilanteessa, joissa jokaisessa käytettiin samoja kahden eri selkeystason puheärsykkeitä (kohinalla ja ilman), mutta joissa koehenkilöiltä edellytettiin erilaisia ärsykkeenjälkeisiä motorisia toimenpiteitä. Yhteensä kymmeneltä koehenkilöltä saadut tulokset, joissa aivoaktivaation lähteitä mallinnettiin yhden dipolin analyysimenetelmällä, osoittavat että "aktivaation painopiste" vasemmassa aivopuoliskossa siirtyi taaemmaksi kuuloaivokuorella havaitsemistilanteiden motoristen toimenpiteiden aktiivisuuden lisääntyessä. Tämä löydös puoltaa ajatusta siitä, että puheen motorisella järjestelmällä on rooli puheen havaitsemisessa. Tämä työ, joka suoritettiin valmistelevana osana laajempaa projektia, luo pohjatyöllään ja tuloksillaan lupaavat lähtökohdat mitä erilaisimpiin puheen havaitsemisen neuraalista luonnetta valottaviin jatkoanalyyseihin.Speech perception refers to a process by which the sounds of language are heard, interpreted and understood. Although this process may seem like a trivial task, the neural mechanisms underlying it are anything but simple. Indeed, the key question puzzling the minds of speech perception researchers is how listeners extract the significant information from the acoustic speech signal? How is the mapping between properties of the acoustic signal and linguistic elements, such as phonemes, done? Distinct theoretical perspectives have been proposed to answer these questions. A crucial distinction among these perspectives can be put in the form of a question: does the speech motor system have a role in speech perception? In this thesis, the role of the speech motor system in speech perception was studied using magnetoencephalography (MEG). The neural mechanisms underlying speech perception was investigated in four different perception conditions, each comprising the same speech stimuli (with two levels of ambiguity) but differing on the subsequent motor task. The results, derived from ten subjects, show a clear shift of the equivalent current dipole (ECD), used in modeling the underlying neuronal sources, to a more posterior position in the left hemisphere with the more active subsequent-to-stimuli motor tasks. This suggests that the motor system does indeed have a role in speech perception. The outcome of this thesis, which was conducted as a preliminary part of a larger project, serves as a promising basis for further study on speech perception, providing the necessary groundwork which allows more refined analyses to take place

From Introspection to Essence: The Auditory Nature of Inner Speech

To some it is a shallow platitude that inner speech always has an auditory-phonological component. To others, it is an empirical hypothesis with accumulating support. To yet others it is a false dogma. In this chapter, I defend the claim that inner speech always has an auditory-phonological component, confining the claim to adults with ordinary speech and hearing. It is one thing, I emphasize, to assert that inner speech often, or even typically, has an auditory-phonological component—quite another to propose that it always does. When forced to argue for the stronger point, we stand to make a number of interesting discoveries about inner speech itself, and about our means for discriminating it from other psycholinguistic phenomena. Establishing the stronger conclusion also provides new leverage on debates concerning how we should conceive of, diagnose, and explain auditory verbal hallucinations and “inserted thoughts” in schizophrenia

CORTICAL DYNAMICS OF AUDITORY-VISUAL SPEECH: A FORWARD MODEL OF MULTISENSORY INTEGRATION.

In noisy settings, seeing the interlocutor's face helps to disambiguate what is being said. For this to happen, the brain must integrate auditory and visual information. Three major problems are (1) bringing together separate sensory streams of information, (2) extracting auditory and visual speech information, and (3) identifying this information as a unified auditory-visual percept. In this dissertation, a new representational framework for auditory visual (AV) speech integration is offered. The experimental work (psychophysics and electrophysiology (EEG)) suggests specific neural mechanisms for solving problems (1), (2), and (3) that are consistent with a (forward) 'analysis-by-synthesis' view of AV speech integration. In Chapter I, multisensory perception and integration are reviewed. A unified conceptual framework serves as background for the study of AV speech integration. In Chapter II, psychophysics testing the perception of desynchronized AV speech inputs show the existence of a ~250ms temporal window of integration in AV speech integration. In Chapter III, an EEG study shows that visual speech modulates early on the neural processing of auditory speech. Two functionally independent modulations are (i) a ~250ms amplitude reduction of auditory evoked potentials (AEPs) and (ii) a systematic temporal facilitation of the same AEPs as a function of the saliency of visual speech. In Chapter IV, an EEG study of desynchronized AV speech inputs shows that (i) fine-grained (gamma, ~25ms) and (ii) coarse-grained (theta, ~250ms) neural mechanisms simultaneously mediate the processing of AV speech. In Chapter V, a new illusory effect is proposed, where non-speech visual signals modify the perceptual quality of auditory objects. EEG results show very different patterns of activation as compared to those observed in AV speech integration. An MEG experiment is subsequently proposed to test hypotheses on the origins of these differences. In Chapter VI, the 'analysis-by-synthesis' model of AV speech integration is contrasted with major speech theories. From a Cognitive Neuroscience perspective, the 'analysis-by-synthesis' model is argued to offer the most sensible representational system for AV speech integration. This thesis shows that AV speech integration results from both the statistical nature of stimulation and the inherent predictive capabilities of the nervous system

Spoonerisms: An Analysis of Language Processing in Light of Neurobiology

Spoonerisms are described as the category of speech errors involving jumbled-up words. The author examines language, the brain, and the correlation between spoonerisms and the neural structures involved in language processing

Recognizing Speech in a Novel Accent: The Motor Theory of Speech Perception Reframed

The motor theory of speech perception holds that we perceive the speech of another in terms of a motor representation of that speech. However, when we have learned to recognize a foreign accent, it seems plausible that recognition of a word rarely involves reconstruction of the speech gestures of the speaker rather than the listener. To better assess the motor theory and this observation, we proceed in three stages. Part 1 places the motor theory of speech perception in a larger framework based on our earlier models of the adaptive formation of mirror neurons for grasping, and for viewing extensions of that mirror system as part of a larger system for neuro-linguistic processing, augmented by the present consideration of recognizing speech in a novel accent. Part 2 then offers a novel computational model of how a listener comes to understand the speech of someone speaking the listener's native language with a foreign accent. The core tenet of the model is that the listener uses hypotheses about the word the speaker is currently uttering to update probabilities linking the sound produced by the speaker to phonemes in the native language repertoire of the listener. This, on average, improves the recognition of later words. This model is neutral regarding the nature of the representations it uses (motor vs. auditory). It serve as a reference point for the discussion in Part 3, which proposes a dual-stream neuro-linguistic architecture to revisits claims for and against the motor theory of speech perception and the relevance of mirror neurons, and extracts some implications for the reframing of the motor theory