Multimodal imaging of language perception

This Thesis draws together several lines of research by examining language perception in the same individuals using three neuroimaging methods: magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and electroencephalography (EEG). The MEG experiments conducted in this Thesis demonstrated that neural processing of written and spoken words converges to the superior temporal cortex following initial modality-specific analysis. In both reading and speech perception, the superior temporal cortex is involved in processing word meaning at ∼250-450 ms in the left hemisphere and after ∼450 ms bilaterally. The data thus support a view of a largely shared semantic system in auditory and visual language perception, in line with the assumption that reading acquisition makes use of the neural systems originally developed for speech perception during evolution and in individual language development. The MEG experiments on reading morphologically complex words showed that the left superior temporal activation was enhanced for the morphologically complex words at ∼200-700 ms. The results suggest that the majority of inflected words in the highly inflected Finnish language are represented in a decomposed form and that the decomposition process requires additional neural resources. Only very high-frequency inflected words may acquire full-form representations. The MEG results on parafoveal preview in reading indicated that neural processing of written words in the left hemisphere is affected by a preview of words in the right visual field. The underlying neural mechanism may facilitate reading of connected text in natural conditions. In a direct comparison, MEG and fMRI showed diverging activation patterns in a reading task although the same individuals were performing the same task. Based on the similarity of the EEG responses recorded simultaneously with both MEG and fMRI, the participants were performing the task similarly during the two recordings. The divergent MEG and fMRI results cannot be attributed to differences in the experimental procedures or language since these factors were controlled. Rather, they are likely to reflect actual dissimilarities in the way neural activity in a high-level cognitive task is picked up by MEG evoked responses and fMRI signals

Contributions of local speech encoding and functional connectivity to audio-visual speech perception

Seeing a speaker’s face enhances speech intelligibility in adverse environments. We investigated the underlying network mechanisms by quantifying local speech representations and directed connectivity in MEG data obtained while human participants listened to speech of varying acoustic SNR and visual context. During high acoustic SNR speech encoding by temporally entrained brain activity was strong in temporal and inferior frontal cortex, while during low SNR strong entrainment emerged in premotor and superior frontal cortex. These changes in local encoding were accompanied by changes in directed connectivity along the ventral stream and the auditory-premotor axis. Importantly, the behavioral benefit arising from seeing the speaker’s face was not predicted by changes in local encoding but rather by enhanced functional connectivity between temporal and inferior frontal cortex. Our results demonstrate a role of auditory-frontal interactions in visual speech representations and suggest that functional connectivity along the ventral pathway facilitates speech comprehension in multisensory environments

Neural mechanisms underlying word- and phrase-level morphological parsing

How is morphological and morphosyntactic information processed during sentence reading? Are the neural mechanisms underlying word- and phrase-level combinatorial processing overlapping or distinct? Here, electroencephalography (EEG) and magnetoencephalography (MEG) responses were recorded simultaneously during silent reading of Finnish sentences. The experimental conditions included 1) well-formed grammatical sentences (correct condition), 2) sentences containing morphosyntactic violations (adjective–noun number agreement violations), 3) morphological violations (incorrect stem allomorph and inflectional suffix combination), and 4) combined violations, containing both morphosyntactic and morphological violations. Signal space and source modeling results showed that morphosyntactic violations elicited a left anterior negativity effect, generated particularly in the left inferior frontal area. Morphological violations elicited a widespread negativity, resembling the N400. The neural sources of this negativity were localized most prominently to the right temporal cortical networks. Furthermore, all violations elicited P600 effects with similar widespread bilateral fronto-temporal neural generators that did not differ between morphosyntactic and morphological conditions. Our findings suggest at least partially distinct subnetworks in the fronto-temporal cortices for morphological and morphosyntactic parsing during the earlier stages of processes (∼400 ms post stimulus onset) and shared neural generators for the later processing stages.Peer reviewe

Spatiotemporal Dynamics of Modality-Specific and Supramodal Word Processing

AbstractThe ability of written and spoken words to access the same semantic meaning provides a test case for the multimodal convergence of information from sensory to associative areas. Using anatomically constrained magnetoencephalography (aMEG), the present study investigated the stages of word comprehension in real time in the auditory and visual modalities, as subjects participated in a semantic judgment task. Activity spread from the primary sensory areas along the respective ventral processing streams and converged in anterior temporal and inferior prefrontal regions, primarily on the left at around 400 ms. Comparison of response patterns during repetition priming between the two modalities suggest that they are initiated by modality-specific memory systems, but that they are eventually elaborated mainly in supramodal areas

Syntax through the looking glass: A review on two-word linguistic processing across behavioral, neuroimaging and neurostimulation studies

In recent years a growing number of studies on syntactic processing has employed basic two-word constructions (e.g., “the tree”) to characterize the fundamental aspects of linguistic composition. This large body of evidence allows, for the first time, to closely examine which cognitive processes and neural substrates support the combination of two syntactic units into a more complex one, mirroring the nature of combinatory operations described in theoretical linguistics. The present review comprehensively examines behavioural, neuroimaging and neurostimulation studies investigating basic syntactic composition, covering more than forty years of psycho- and neuro-linguistic research. Across several paradigms, four key features of syntactic composition have emerged: (1) the rule-based and (2) automatic nature of the combinatorial process, (3) a central role of Broca’s area and the posterior temporal lobe in representing and combining syntactic features, and (4) the reliance on efficient bottom-up integration rather than top-down prediction

Context-dependent lexical ambiguity resolution: MEG evidence for the time-course of activity in left inferior frontal gyrus and posterior middle temporal gyrus

An MEG study investigated the role of context in semantic interpretation by examining the comprehension of ambiguous words in contexts leading to different interpretations. We compared high-ambiguity words in minimally different contexts (to bowl, the bowl) to low-ambiguity counterparts (the tray, to flog). Whole brain beamforming revealed the engagement of left inferior frontal gyrus (LIFG) and posterior middle temporal gyrus (LPMTG). Points of interest analyses showed that both these sites showed a stronger response to verb-contexts by 200 ms post-stimulus and displayed overlapping ambiguity effects that were sustained from 300 ms onwards. The effect of context was stronger for high-ambiguity words than for low-ambiguity words at several different time points, including within the first 100 ms post-stimulus. Unlike LIFG, LPMTG also showed stronger responses to verb than noun contexts in low-ambiguity trials. We argue that different functional roles previously attributed to LIFG and LPMTG are in fact played out at different periods during processing

Semantic radical consistency and character transparency effects in Chinese: an ERP study

BACKGROUND: This event-related potential (ERP) study aims to investigate the representation and temporal dynamics of Chinese orthography-to-semantics mappings by simultaneously manipulating character transparency and semantic radical consistency. Character components, referred to as radicals, make up the building blocks used dur...postprin

A MEG study of the neural substrates of semantic processing in semantic variant primary progressive aphasia

Despite a well-documented pattern of semantic memory (SM) impairment, the patterns of brain activation during semantic processing in svPPA still remain poorly understood. The current study aimed to investigate the neural substrates of residual semantic processing in the context of this significant but selective SM impairment, through the case study of one svPPA patient. One svPPA patient (EC) and six elderly controls carried out a general-level semantic categorization task (biological and manufactured objects) while their brain activity was recorded using magnetoencephalography (MEG). Despite similar behavioral performance, EC showed hyperactivation of the left inferior temporal gyrus (ITG) and right anterior temporal lobe (ATL) relative to controls. This suggests that periatrophic regions within the ATL region may support preserved semantic abilities in svPPA. These results thus contribute to our understanding of the brain regions which are recruited to compensate for bilateral atrophy of the ATL and ensure residual semantic processing in svPPA