Disentangling meaning in the brain: Left temporal involvement in agreement processing

Published online 18 November 2016Sentence comprehension is successfully accomplished by means of a form-to-meaning mapping procedure that relies on the extraction of morphosyntactic information from the input and its mapping to higher-level semantic–discourse representations. In this study, we sought to determine whether neuroanatomically distinct brain regions are involved in the processing of different types of information contained in the propositional meaning of a sentence, namely person and number. While person information indexes the role that an individual has in discourse (i.e., the speaker, the addressee or the entity being talked about by speaker and addressee), number indicates its cardinality (i.e., a single entity vs a multitude of entities). An event-related functional magnetic resonance imaging (fMRI) experiment was run using agreement-Correct and Person- and Number-violated sentences in Spanish, to disentangle the processing mechanisms and neural substrates associated with the building of discourse and cardinality representations. The contrast between Person and Number Violations showed qualitative and quantitative differences. A greater response for person compared to number was found in the left middle temporal gyrus (LMTG). However, critically, a posterior-to-anterior functional gradient emerged within this region. While the posterior portion of the LMTG was sensitive to both Person and Number Violations, the anterior portion of this region showed selective response for Person Violations. These results confirm that the comprehension of the propositional meaning of a sentence results from a composite, feature-sensitive mechanism of form-to-meaning mapping in which the nodes of the language network are differentially involved.BCBL acknowledges funding from Ayuda Centro de Excelencia Severo Ochoa SEV-2015-0490. S.M. acknowledges funding from the Gipuzkoako Foru Aldundia Fellowship Program and from grant PI_2014_38 from the Basque Government. N.M. was funded by grant PSI2012- 32350 and PSI2015-65694-P from the Spanish Ministry of Economy and Competitiveness. M.C was funded by grant PSI2012-31448 from the Spanish Ministry of Science and Innovation and ERC-2011-ADG-295362 from the European Research Council

Resolving Semantic Ambiguities in Sentences: Cognitive Processes and Brain Mechanisms

fMRI studies of how the brain processes sentences containing semantically ambiguous words have consistently implicated (i) the left inferior frontal gyrus (LIFG) and (ii) posterior regions of the left temporal lobe in processing high-ambiguity sentences. This article reviews recent findings on this topic and relates them to (i) psycholinguistic theories about the underlying cognitive processes and (ii) general neuro-cognitive accounts of the relevant brain regions. We suggest that the LIFG plays a general role in the cognitive control process that are necessary to select contextually relevant meanings and to reinterpret sentences that were initially misunderstood, but it is currently unclear whether these control processes should best be characterised in terms of specific processes such as conflict resolution and controlled retrieval that are required for high-ambiguity sentences, or whether its function is better characterised in terms of a more general set of ‘unification’ processes. In contrast to the relatively rapid progress that has been made in understanding the function of the LIFG, we suggest that the contribution of the posterior temporal lobe is less well understood and future work is needed to clarify its role in sentence comprehension

From minimal dependencies to sentence contexts: neural correlates of agreement processing

289 p.Language comprehension is incremental, involving the integration of formal and conceptual information from different words, together with the need to resolve conflicting cues when unexpected information occurs. However, despite the extensive amount of findings regarding how the brain deals with these information, two essential and still open questions are 1whether the neural circuit(s) for coding syntactic and semantic information embedded in our linguistic code are the same or different, and 2whether the possible interaction(s) between these two different types of information leaves a trace in the brain response. The current thesis seeks to segregate the neuro-anatomical substrates of these processes by taking advantage of the Spanish agreement system. This system comprised those procedural mechanisms concerning the regular assignment of the number [singular, plural], person [first, second and third] and/or gender [feminine, masculine] information, associated with different sentence constituents. Experimental manipulations concerning different agreement features and the elements involved in an agreement relation, allowed us to characterize the neural network underlying agreement processing. This thesis comprised five experiments: while experiments I and II explored nominal dependencies in local as well as non-local relations, experiments III, IV and V explored subject-verb relations in a more complex sentence context. To distinguish between purely syntactic mechanisms and those where semantic and syntactic factors would interact during language comprehension, different types of agreement relations and/or agreement features were manipulated in well- and ill-formed constructions. The interaction effect between the different factors included in each experiment was always the critical comparison. In general, our results include firstly a functional dissociation between well-formed and ill-formed constructions: while ill-formed constructions recruited a bilateral distributed fronto-parietal network associated to conflict monitoring operations, not language specific, well-formed constructions recruited a left lateralized fronto-temporo-parietal network that seems to be specifically related to different aspects of phrase and sentence processing. Secondly, there was an anterior to posterior functional gradient associated to the middle and superior temporal cortex that consistently appears across experiments. Specifically, while the posterior portion of the left MTG-STG seems to be related to the storage and retrieval of lexical and morpho-syntactic information, the anterior portion of this region was related to syntactic-combinatorial building mechanisms. Critically, in the most anterior part of the left temporal cortex, corresponding with the middle and superior temporal pole, form-to-meaning mapping processes seems to be represented. Thirdly, the response of the left temporal cortex appears to be controlled by left inferior frontal regions (LIFG). Finally, left parietal regions such us the angular gyrus showed increased activation for those manipulations involving semantic factors (e.g., conceptual gender and Unagreement constructions), highlighting its crucial role in the processing of different types of semantic information (e.g., conceptual integration and semantic-discourse integration). Overall, these findings highlight the sensitivity of the agreement system to syntactic and semantic factors embedded into an agreement relation, opening new windows to the study of agreement computation and language comprehension.bcbl: basque center on cognition, brain and languag

From minimal dependencies to sentence contexts: neural correlates of agreement processing

289 p.Language comprehension is incremental, involving the integration of formal and conceptual information from different words, together with the need to resolve conflicting cues when unexpected information occurs. However, despite the extensive amount of findings regarding how the brain deals with these information, two essential and still open questions are 1whether the neural circuit(s) for coding syntactic and semantic information embedded in our linguistic code are the same or different, and 2whether the possible interaction(s) between these two different types of information leaves a trace in the brain response. The current thesis seeks to segregate the neuro-anatomical substrates of these processes by taking advantage of the Spanish agreement system. This system comprised those procedural mechanisms concerning the regular assignment of the number [singular, plural], person [first, second and third] and/or gender [feminine, masculine] information, associated with different sentence constituents. Experimental manipulations concerning different agreement features and the elements involved in an agreement relation, allowed us to characterize the neural network underlying agreement processing. This thesis comprised five experiments: while experiments I and II explored nominal dependencies in local as well as non-local relations, experiments III, IV and V explored subject-verb relations in a more complex sentence context. To distinguish between purely syntactic mechanisms and those where semantic and syntactic factors would interact during language comprehension, different types of agreement relations and/or agreement features were manipulated in well- and ill-formed constructions. The interaction effect between the different factors included in each experiment was always the critical comparison. In general, our results include firstly a functional dissociation between well-formed and ill-formed constructions: while ill-formed constructions recruited a bilateral distributed fronto-parietal network associated to conflict monitoring operations, not language specific, well-formed constructions recruited a left lateralized fronto-temporo-parietal network that seems to be specifically related to different aspects of phrase and sentence processing. Secondly, there was an anterior to posterior functional gradient associated to the middle and superior temporal cortex that consistently appears across experiments. Specifically, while the posterior portion of the left MTG-STG seems to be related to the storage and retrieval of lexical and morpho-syntactic information, the anterior portion of this region was related to syntactic-combinatorial building mechanisms. Critically, in the most anterior part of the left temporal cortex, corresponding with the middle and superior temporal pole, form-to-meaning mapping processes seems to be represented. Thirdly, the response of the left temporal cortex appears to be controlled by left inferior frontal regions (LIFG). Finally, left parietal regions such us the angular gyrus showed increased activation for those manipulations involving semantic factors (e.g., conceptual gender and Unagreement constructions), highlighting its crucial role in the processing of different types of semantic information (e.g., conceptual integration and semantic-discourse integration). Overall, these findings highlight the sensitivity of the agreement system to syntactic and semantic factors embedded into an agreement relation, opening new windows to the study of agreement computation and language comprehension.bcbl: basque center on cognition, brain and languag

Naturalistic language comprehension : a fMRI study on semantics in a narrative context

Semantiikka tutkii kieleen sisältyviä merkityksiä, joita tarvitaan kielen ymmärryksessä. Kuinka aivomme käsittelevät semantiikkaa ja kuinka ymmärrämme erityisesti luonnollisessa muodossa olevaa kieltä, on vielä aivotutkijoille epäselvää. Tässä tutkimuksessa kysyttiin, miten laajemmassa kontekstissa, narratiivissa, olevan kielen ymmärrys ja semanttinen prosessointi heijastuu aivojen aktiivisuuteen. Koehenkilöt kuulivat narratiivin toiminnallisen magneettiresonanssikuvantamisen (fMRI) aikana. Narratiivin semanttinen sisältö mallinnettiin laskennallisesti word2vec algoritmin avulla, ja tätä mallia verrattiin veren happitasosta riippuvaiseen (BOLD) aivosignaaliin ridge regression avulla vokseli kerrallaan. Lähestymistavalla saatiin eristettyä yksityiskohtaisempaa tietoa jatkuvan stimuluksen aivodatasta perustuen kielen semanttiseen sisältöön. Subjektien välinen BOLD-signaalin korrelaatio (ISC) itsessään paljasti molempien aivopuoliskojen osallistuvan kielen ymmärrykseen laajasti. Alueellista päällekkäisyyttä löytyi muiden aivoverkostojen kanssa, jotka vastaavat mm. mentalisaatiosta, muistista ja keskittymiskyvystä, mikä viittaa kielen ymmärryksen vaativan myös muiden kognition osien toimintaa. Ridge regression tulokset viittaavat bilateraalisten pikkuaivojen, superiorisen, keskimmäisen sekä mediaalisen etuaivokuoren poimujen, inferiorisen ja mediaalisen parietaalikuoren sekä visuaalikuoren, sekä oikean temporaalikuoren osallistuvan narratiivin semanttiseen prosessointiin aivoissa. Aiempi semantiikan tutkimus on tuottanut samankaltaisia tuloksia, joten word2vec vaikuttaisi tämän tutkimuksen perusteella mallintavan semantiikkaa riittävän hyvin aivotutkimuksen tarpeisiin. Tutkimuksen perusteella molemmat aivopuoliskot osallistuvat kielen laajemman kontekstin käsittelyyn, ja semantiikka nähdään aktivaationa eri puolilla aivokuorta. Nämä aktiivisuudet ovat mahdollisesti riippuvaisia kielen sisällöstä, mutta miten paljon kielen sisältö vaikuttaa eri aivoalueiden osallistumiseen kielen semanttisessa prosessoinnissa, on vielä avoin tutkimuskysymys.Semantics is a study of meaning in language and basis for language comprehension. How these phenomena are processed in the brain is still unclear especially in naturalistic context. In this study, naturalistic language comprehension, and how semantic processing in a narrative context is reflected in brain activity were investigated. Subjects were measured with functional magnetic resonance imaging (fMRI) while listening to a narrative. The semantic content of the narrative was modelled computationally with word2vec and compared to voxel-wise blood-oxygen-level dependent (BOLD) brain signal time courses using ridge regression. This approach provides a novel way to extract more detailed information from the brain data based on semantic content of the stimulus. Inter-subject correlation (ISC) of voxel-wise BOLD signals alone showed both hemispheres taking part in language comprehension. Areas involved in this task overlapped with networks of mentalisation, memory and attention suggesting comprehension requiring other modalities of cognition for its function. Ridge regression suggested cerebellum, superior, middle and medial frontal, inferior and medial parietal and visual cortices bilaterally and temporal cortex on right hemisphere having a role in semantic processing of the narrative. As similar results have been found in previous research on semantics, word2vec appears to model semantics sufficiently and is an applicable tool in brain research. This study suggests contextual language recruiting brain areas in both hemispheres and semantic processing showing as distributed activity on the cortex. This activity is likely dependent on the content of language, but further studies are required to distinguish how strongly brain activity is affected by different semantic contents

Broca and wernicke are dead, or moving past the classic model of language neurobiology

With the advancement of cognitive neuroscience and neuropsychological research, the field of language neurobiology is at a cross-roads with respect to its framing theories. The central thesis of this article is that the major historical framing model, the Classic “Wernicke-Lichtheim-Geschwind” model, and associated terminology, is no longer adequate for contemporary investigations into the neurobiology of language. We argue that the Classic model (1) is based on an outdated brain anatomy; (2) does not adequately represent the distributed connectivity relevant for language, (3) offers a modular and “language centric” perspective, and (4) focuses on cortical structures, for the most part leaving out subcortical regions and relevant connections. To make our case, we discuss the issue of anatomical specificity with a focus on the contemporary usage of the terms “Broca’s and Wernicke’s area”, including results of a survey that was conducted within the language neurobiology community. We demonstrate that there is no consistent anatomical definition of “Broca’s and Wernicke’s Areas”, and propose to replace these terms with more precise anatomical definitions. We illustrate the distributed nature of the language connectome, which extends far beyond the single-pathway notion of arcuate fasciculus connectivity established in the Geschwind’s version of the Classic Model. By illustrating the definitional confusion surrounding “Broca’s and Wernicke’s areas”, and the by illustrating the difficulty integrating the emerging literature on perisylvian white matter connectivity into this model, we hope to expose the limits of the model, argue for its obsolescence, and suggest a path forward in defining a replacement

Neural basis of speech-gesture mismatch detection in schizophrenia spectrum disorders

Background: Patients suffering from schizophrenia spectrum disorders experience the grave effects of their illness on various facets of their daily lives. Previous investigations have shown that schizophrenia spectrum disorder patients have deficits in the perception and recognition of speech accompanied by gestures. In particular, they struggle to differentiate between related and unrelated speech-gesture combinations. Also, patients have considerable difficulties in understanding and processing abstract semantic information. A key region in the integration of speech and gesture is the inferior frontal gyrus embedded in a frontotemporal network, however, it is unclear which neural mechanisms contribute to defective mismatch and abstractness perception during the mismatch detection task. Objective: This study aimed to investigate the neural underpinnings of impaired speech-gesture mismatch detection and abstract semantic processing in schizophrenia spectrum disorder patients and to identify relevant dysfunctional brain areas. Methods: A novel mismatch-detection fMRI paradigm was implemented manipulating speech-gesture abstractness (abstract/concrete) and relatedness (related/unrelated). During fMRI data acquisition, 42 patients (schizophrenia, schizoaffective disorder or other non-organic psychotic disorder [ICD-10: F20, F25, F28; DSM-IV: 295.X]) and 36 healthy controls were presented with short video clips of an actor reciting abstract or concrete sentences accompanied by either a semantically related or unrelated gesture. Participants indicated via button press whether they perceived each gesture as matching the speech content or not. We compared task performances across groups and semantic context (abstract/concrete) using the detection rate from Signal Detection Theory by repeated-measures ANOVA. For the functional MRI data, an event-related design was chosen to measure the hemodynamic responses to each presented video. The data were loaded into a flexible-factorial analysis in a 2 x 2 x 2 design (group x abstractness x relatedness). Between-group conjunctions and group differences were respectively calculated for the contrasts unrelated ] related and abstract ] concrete in whole-brain analyses. Results: Speech-gesture mismatch detection performance was significantly impaired in patients compared to controls, irrespective of abstractness. fMRI data analysis revealed that patients exhibited reduced engagement of the right supplementary motor area and bilateral anterior cingulate cortices for unrelated ] related stimuli. A rostral part of the supplementary motor area was equally activated in both groups. In contrast, we found frontotemporal hyperactivation in patients for the same contrast. Furthermore, patients showed lower activation in bilateral frontal areas including the inferior frontal gyrus for all abstract ] concrete speech-gesture pairs. The temporal lobe, however, was engaged in both groups equally for this contrast. Discussion: In this study, we found evidence for impaired gesture-speech relatedness judgment in schizophrenia spectrum disorders. This was accompanied by dysfunctions of the supplementary motor area and the anterior cingulate cortices, possibly reflecting reduced facilitation of comprehension and defective error processing for unrelated speech-gesture combinations. The frontotemporal hyperactivation may represent an increased processing effort to compensate for the dysfunction. In addition, our data confirmed the conjecture of an inferior frontal gyrus dysfunction contributing to impaired processing of abstract semantic stimuli. Partially intact processing was discovered in a rostral part of the supplementary motor area for mismatches, and in the temporal lobes for abstract stimuli. These findings suggest that semantic processing in schizophrenia spectrum disorders is not completely dysfunctional, but that there is a functioning basis on which therapeutic measures can build on. Conclusion: We provide first evidence that impaired speech-gesture mismatch detection in schizophrenia spectrum disorders could be the result of dysfunctional activation of the supplementary motor area and anterior cingulate cortex. Failure to activate the left inferior frontal gyrus disrupts the integration of abstract speech-gesture combinations in particular. Future investigations should focus on brain-stimulation of these regions to improve communication and social functioning in schizophrenia spectrum disorders

Minimal phrase composition revealed by intracranial recordings

The ability to comprehend phrases is an essential integrative property of the brain. Here we evaluate the neural processes that enable the transition from single word processing to a minimal compositional scheme. Previous research has reported conflicting timing effects of composition, and disagreement persists with respect to inferior frontal and posterior temporal contributions. To address these issues, 19 patients (10 male, 19 female) implanted with penetrating depth or surface subdural intracranial electrodes heard auditory recordings of adjective-noun, pseudoword-noun and adjective-pseudoword phrases and judged whether the phrase matched a picture. Stimulus-dependent alterations in broadband gamma activity, low frequency power and phase-locking values across the language-dominant left hemisphere were derived. This revealed a mosaic located on the lower bank of the posterior superior temporal sulcus (pSTS), in which closely neighboring cortical sites displayed exclusive sensitivity to either lexicality or phrase structure, but not both. Distinct timings were found for effects of phrase composition (210–300 ms) and pseudoword processing (approximately 300–700 ms), and these were localized to neighboring electrodes in pSTS. The pars triangularis and temporal pole encoded anticipation of composition in broadband low frequencies, and both regions exhibited greater functional connectivity with pSTS during phrase composition. Our results suggest that the pSTS is a highly specialized region comprised of sparsely interwoven heterogeneous constituents that encodes both lower and higher level linguistic features. This hub in pSTS for minimal phrase processing may form the neural basis for the human-specific computational capacity for forming hierarchically organized linguistic structures