Multimodal imaging of language perception

This Thesis draws together several lines of research by examining language perception in the same individuals using three neuroimaging methods: magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and electroencephalography (EEG). The MEG experiments conducted in this Thesis demonstrated that neural processing of written and spoken words converges to the superior temporal cortex following initial modality-specific analysis. In both reading and speech perception, the superior temporal cortex is involved in processing word meaning at ∼250-450 ms in the left hemisphere and after ∼450 ms bilaterally. The data thus support a view of a largely shared semantic system in auditory and visual language perception, in line with the assumption that reading acquisition makes use of the neural systems originally developed for speech perception during evolution and in individual language development. The MEG experiments on reading morphologically complex words showed that the left superior temporal activation was enhanced for the morphologically complex words at ∼200-700 ms. The results suggest that the majority of inflected words in the highly inflected Finnish language are represented in a decomposed form and that the decomposition process requires additional neural resources. Only very high-frequency inflected words may acquire full-form representations. The MEG results on parafoveal preview in reading indicated that neural processing of written words in the left hemisphere is affected by a preview of words in the right visual field. The underlying neural mechanism may facilitate reading of connected text in natural conditions. In a direct comparison, MEG and fMRI showed diverging activation patterns in a reading task although the same individuals were performing the same task. Based on the similarity of the EEG responses recorded simultaneously with both MEG and fMRI, the participants were performing the task similarly during the two recordings. The divergent MEG and fMRI results cannot be attributed to differences in the experimental procedures or language since these factors were controlled. Rather, they are likely to reflect actual dissimilarities in the way neural activity in a high-level cognitive task is picked up by MEG evoked responses and fMRI signals

Information properties of morphologically complex words modulate brain activity during word reading

Neuroimaging studies of the reading process point to functionally distinct stages in word recognition. Yet, current understanding of the operations linked to those various stages is mainly descriptive in nature. Approaches developed in the field of computational linguistics may offer a more quantitative approach for understanding brain dynamics. Our aim was to evaluate whether a statistical model of morphology, with well-defined computational principles, can capture the neural dynamics of reading, using the concept of surprisal from information theory as the common measure. The Morfessor model, created for unsupervised discovery of morphemes, is based on the minimum description length principle and attempts to find optimal units of representation for complex words. In a word recognition task, we correlated brain responses to word surprisal values derived from Morfessor and from other psycholinguistic variables that have been linked with various levels of linguistic abstraction. The magnetoencephalography data analysis focused on spatially, temporally and functionally distinct components of cortical activation observed in reading tasks. The early occipital and occipito-temporal responses were correlated with parameters relating to visual complexity and orthographic properties, whereas the later bilateral superior temporal activation was correlated with whole-word based and morphological models. The results show that the word processing costs estimated by the statistical Morfessor model are relevant for brain dynamics of reading during late processing stages.Peer reviewe

Neural dynamics of inflectional and derivational processing in spoken word comprehension: laterality and automaticity.

Rapid and automatic processing of grammatical complexity is argued to take place during speech comprehension, engaging a left-lateralized fronto-temporal language network. Here we address how neural activity in these regions is modulated by the grammatical properties of spoken words. We used combined magneto- and electroencephalography to delineate the spatiotemporal patterns of activity that support the recognition of morphologically complex words in English with inflectional (-s) and derivational (-er) affixes (e.g., bakes, baker). The mismatch negativity, an index of linguistic memory traces elicited in a passive listening paradigm, was used to examine the neural dynamics elicited by morphologically complex words. Results revealed an initial peak 130-180 ms after the deviation point with a major source in left superior temporal cortex. The localization of this early activation showed a sensitivity to two grammatical properties of the stimuli: (1) the presence of morphological complexity, with affixed words showing increased left-laterality compared to non-affixed words; and (2) the grammatical category, with affixed verbs showing greater left-lateralization in inferior frontal gyrus compared to affixed nouns (bakes vs. beaks). This automatic brain response was additionally sensitive to semantic coherence (the meaning of the stem vs. the meaning of the whole form) in left middle temporal cortex. These results demonstrate that the spatiotemporal pattern of neural activity in spoken word processing is modulated by the presence of morphological structure, predominantly engaging the left-hemisphere's fronto-temporal language network, and does not require focused attention on the linguistic input

Professional or amateur? The phonological output buffer as a working memory operator

The Phonological Output Buffer (POB) is thought to be the stage in language production where phonemes are held in working memory and assembled into words. The neural implementation of the POB remains unclear despite a wealth of phenomenological data. Individuals with POB impairment make phonological errors when they produce words and non-words, including phoneme omissions, insertions, transpositions, substitutions and perseverations. Errors can apply to different kinds and sizes of units, such as phonemes, number words, morphological affixes, and function words, and evidence from POB impairments suggests that units tend to substituted with units of the same kind-e.g., numbers with numbers and whole morphological affixes with other affixes. This suggests that different units are processed and stored in the POB in the same stage, but perhaps separately in different mini-stores. Further, similar impairments can affect the buffer used to produce Sign Language, which raises the question of whether it is instantiated in a distinct device with the same design. However, what appear as separate buffers may be distinct regions in the activity space of a single extended POB network, connected with a lexicon network. The self-consistency of this idea can be assessed by studying an autoassociative Potts network, as a model of memory storage distributed over several cortical areas, and testing whether the network can represent both units of word and signs, reflecting the types and patterns of errors made by individuals with POB impairment

Spatiotemporal Dynamics of the Processing of Spoken Inflected and Derived Words: A Combined EEG and MEG Study

The spatiotemporal dynamics of the neural processing of spoken morphologically complex words are still an open issue. In the current study, we investigated the time course and neural sources of spoken inflected and derived words using simultaneously recorded electroencephalography (EEG) and magnetoencephalography (MEG) responses. Ten participants (native speakers) listened to inflected, derived, and monomorphemic Finnish words and judged their acceptability. EEG and MEG responses were time-locked to both the stimulus onset and the critical point (suffix onset for complex words, uniqueness point for monomorphemic words). The ERP results showed that inflected words elicited a larger left-lateralized negativity than derived and monomorphemic words approximately 200 ms after the critical point. Source modeling of MEG responses showed one bilateral source in the superior temporal area ∼100 ms after the critical point, with derived words eliciting stronger source amplitudes than inflected and monomorphemic words in the right hemisphere. Source modeling also showed two sources in the temporal cortex approximately 200 ms after the critical point. There, inflected words showed a more systematic pattern in source locations and elicited temporally distinct source activity in comparison to the derived word condition. The current results provide electrophysiological evidence for at least partially distinct cortical processing of spoken inflected and derived words. In general, the results support models of morphological processing stating that during the recognition of inflected words, the constituent morphemes are accessed separately. With regard to derived words, stem and suffix morphemes might be at least initially activated along with the whole word representation

Contributions to the functional neuroanatomy of morphosyntactic processing in L2

Studies about bilingualism and second language acquisition (SLA) have a long tradition within linguistic and psycholinguistic research. With the global population becoming more and more multilingual and the recent proliferation of research in cognitive neuroscience, an increasing number of studies examining the way our brain is able to learn, represent, and handle more than one language at the same time are currently available. However, few attempts have been made to transpose psycholinguistic models of SLA into functional neuroanatomic models. An important problem that arises when pursuing this goal is partially due to the delay in the development of cognitive neuroscience of language compared to psycholinguistics. In general, neurolinguistic models focus on very broad and general questions about bilingualism, whereas psycholinguistic research is already at the stage of addressing more specific and fine-tuned questions. This Granularity Mismatch Problem (Poeppel & Embick, 2005) in the degree of zooming into this research topic is not exclusive of L2 research, but it is present in language research in general (Hauser & Bever, 2008). In either case, it often becomes difficult to put together the results from these different perspectives into one integrated model

Prosodically controlled derivations in the mental lexicon

Swedish morphemes are classified as prosodically specified or prosodically unspecified, depending on lexical or phonological stress, respectively. Here, we investigate the allomorphy of the suffix -(i)sk, which indicates the distinction between lexical and phonological stress; if attached to a lexically stressed morpheme, it takes a non-syllabic form (-sk), whereas if attached to a phonologically stressed morpheme, an epenthetic vowel is inserted (-isk). Using mismatch negativity (MMN), we explored the neural processing of this allomorphy across lexically stressed and phonologically stressed morphemes. In an oddball paradigm, participants were occasionally presented with congruent and incongruent derivations, created by the suffix -(i)sk, within the repetitive presentation of their monomorphemic stems. The results indicated that the congruent derivation of the lexically stressed stem elicited a larger MMN than the incongruent sequences of the same stem and the derivational suffix, whereas after the phonologically stressed stem a non-significant tendency towards an opposite pattern was observed. We argue that the significant MMN response to the congruent derivation in the lexical stress condition is in line with lexical MMN, indicating a holistic processing of the sequence of lexically stressed stem and derivational suffix. The enhanced MMN response to the incongruent derivation in the phonological stress condition, on the other hand, is suggested to reflect combinatorial processing of the sequence of phonologically stressed stem and derivational suffix. These findings bring a new aspect to the dual-system approach to neural processing of morphologically complex words, namely the specification of word stress.Peer reviewe