The neural basis of sign language processing in deaf signers: An activation likelihood estimation meta-analysis

The neurophysiological response during processing of sign language (SL) has been studied since the advent of Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI). Nevertheless, the neural substrates of SL remain subject to debate, especially with regard to involvement and relative lateralization of SL processing without production in (left) inferior frontal gyrus (IFG; e.g., Campbell, MacSweeney, & Waters, 2007; Emmorey, 2006, 2015). Our present contribution is the first to address these questions meta-analytically, by exploring functional convergence on the whole-brain level using previous fMRI and PET studies of SL processing in deaf signers. We screened 163 records in PubMed and Web of Science to identify studies of SL processing in deaf signers conducted with fMRI or PET that reported foci data for one of the two whole-brain contrasts: (1) “SL processing vs. control” or (2) “SL processing vs. low-level baseline”. This resulted in a total of 21 studies reporting 23 experiments matching our selection criteria. We manually extracted foci data and performed a coordinate-based Activation Likelihood Estimation (ALE) analysis using GingerALE (Eickhoff et al., 2009). Our selection criteria and the ALE method allow us to identify regions that are consistently involved in processing SL across studies and tasks. Our analysis reveals that processing of SL stimuli of varying linguistic complexity engages widely distributed bilateral fronto-occipito-temporal networks in deaf signers. We find significant clusters in both hemispheres, with the largest cluster (5240 mm3) being located in left IFG, spanning Broca’s region (posterior BA 45 and the dorsal portion of BA 44). Other clusters are located in right middle and inferior temporal gyrus (BA 37), right IFG (BA 45), left middle occipital gyrus (BA 19), right superior temporal gyrus (BA 22), left precentral and middle frontal gyrus (BA 6 and 8), as well as left insula (BA 13). On these clusters, we calculated lateralization indices using hemispheric and anatomical masks: SL comprehension is slightly left-lateralized globally, and strongly left-lateralized in Broca’s region. Sub-regionally, left-lateralization is strongest in BA 44 (Table 1). Next, we performed a contrast analysis between SL and an independent dataset of action observation in hearing non-signers (Papitto, Friederici, & Zaccarella, 2019) to determine which regions are associated with processing of human actions and movements irrespective of the presence of linguistic information. Only studies of observation of non-linguistic manual actions were included in the final set (n = 26), for example, excluding the handling of objects. Significant clusters involved in the linguistic aspects of SL comprehension were found in left Broca’s region (centered in dorsal BA 44), right superior temporal gyrus (BA 22), and left middle frontal and precentral gyrus (BA 6 and 8; Figure 1A, B, D and E). Meta-analytic connectivity modelling for the surviving cluster in Broca’s region using the BrainMap database then revealed that it is co-activated with the classical language network and functionally primarily associated with cognition and language processing (Figure 1C and D). In line with studies of spoken and written language processing (Zaccarella, Schell, & Friederici, 2017; Friederici, Chomsky, Berwick, Moro, & Bolhuis, 2017), our meta-analysis points to Broca’s region and especially left BA 44 as a hub in the language network that is involved in language processing independent of modality. Right IFG activity is not language-specific but may be specific to the visuo-gestural modality (Campbell et al., 2007). References Amunts, K., Schleicher, A., Bürgel, U., Mohlberg, H., Uylings, H. B., & Zilles, K. (1999). Broca’s region revisited: Cytoarchitecture and intersubject variability. The Journal of Comparative Neurology, 412(2), 319-341. Campbell, R., MacSweeney, M., & Waters, D. (2007). Sign language and the brain: A review. Journal of Deaf Studies and Deaf Education, 13(1), 3-20. doi: 10.1093/deafed/enm035 Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009). Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: A random-effects approach based on empirical estimates of spatial uncertainty. Human Brain Mapping, 30(9), 2907-2926. doi: 10.1002/hbm.20718 Emmorey, K. (2006). The role of Broca’s area in sign language. In Y. Grodzinsky & K. Amunts (Eds.), Broca’s region (p. 169-184). Oxford, England: Oxford UP. Emmorey, K. (2015). The neurobiology of sign language. In A. W. Toga, P. Bandettini, P. Thompson, & K. Friston (Eds.), Brain mapping: An encyclopedic reference (Vol. 3, p. 475-479). London, England: Academic Press. doi: 10.1016/B978-0-12-397025-1.00272-4 Friederici, A. D., Chomsky, N., Berwick, R. C., Moro, A., & Bolhuis, J. J. (2017). Language, mind and brain. Nature Human Behaviour. doi: 10.1038/s41562-017-0184-4 Matsuo, K., Chen, S.-H. A., & Tseng, W.-Y. I. (2012). AveLI: A robust lateralization index in functional magnetic resonance imaging using unbiased threshold-free computation. Journal of Neuroscience Methods, 205(1), 119-129. doi: 10.1016/j.jneumeth.2011.12.020 Papitto, G., Friederici, A. D., & Zaccarella, E. (2019). A neuroanatomical comparison of action domains using Activation Likelihood Estimation meta-analysis [Unpublished Manuscript, Max Planck Institute for Human Cognitive & Brain Sciences]. Leipzig, Germany. Zaccarella, E., Schell, M., & Friederici, A. D. (2017). Reviewing the functional basis of the syntactic Merge mechanism for language: A coordinate-based activation likelihood estimation meta-analysis. Neuroscience & Biobehavioral Reviews, 80, 646-656. doi: 10.1016/j.neubiorev.2017.06.01

Syntax through the looking glass: A review on two-word linguistic processing across behavioral, neuroimaging and neurostimulation studies

In recent years a growing number of studies on syntactic processing has employed basic two-word constructions (e.g., “the tree”) to characterize the fundamental aspects of linguistic composition. This large body of evidence allows, for the first time, to closely examine which cognitive processes and neural substrates support the combination of two syntactic units into a more complex one, mirroring the nature of combinatory operations described in theoretical linguistics. The present review comprehensively examines behavioural, neuroimaging and neurostimulation studies investigating basic syntactic composition, covering more than 40 years of psycho- and neuro-linguistic research. Across several paradigms, four key features of syntactic composition have emerged: (1) the rule-based and (2) automatic nature of the combinatorial process, (3) a central role of Broca’s area and the posterior temporal lobe in representing and combining syntactic features, and (4) the reliance on efficient bottom-up integration rather than top-down prediction

Language and action in Broca’s area: Computational differentiation and cortical segregation

Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca’s area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca’s subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world

Differential contributions of left-hemispheric language regions to basic semantic composition

Semantic composition, the ability to combine single words to form complex meanings, is a core feature of human language. Despite growing interest in the basis of semantic composition, the neural correlates and the interaction of regions within this network remain a matter of debate. We designed a well-controlled two-word fMRI paradigm in which phrases only differed along the semantic dimension while keeping syntactic information alike. Healthy participants listened to meaningful (“fresh apple”), anomalous (“awake apple”) and pseudoword phrases (“awake gufel”) while performing an implicit and an explicit semantic task. We identified neural signatures for distinct processes during basic semantic composition. When lexical information is kept constant across conditions and the evaluation of phrasal plausibility is examined (meaningful vs. anomalous phrases), a small set of mostly left-hemispheric semantic regions, including the anterior part of the left angular gyrus, is found active. Conversely, when the load of lexical information—independently of phrasal plausibility—is varied (meaningful or anomalous vs. pseudoword phrases), conceptual combination involves a wide-spread left-hemispheric network comprising executive semantic control regions and general conceptual representation regions. Within this network, the functional coupling between the left anterior inferior frontal gyrus, the bilateral pre-supplementary motor area and the posterior angular gyrus specifically increases for meaningful phrases relative to pseudoword phrases. Stronger effects in the explicit task further suggest task-dependent neural recruitment. Overall, we provide a separation between distinct nodes of the semantic network, whose functional contributions depend on the type of compositional process under analysis

Neural classification maps for distinct word combinations in Broca’s area

Humans are equipped with the remarkable ability to comprehend an infinite number of utterances. Relations between grammatical categories restrict the way words combine into phrases and sentences. How the brain recognises different word combinations remains largely unknown, although this is a necessary condition for combinatorial unboundedness in language. Here, we used functional magnetic resonance imaging and multivariate pattern analysis to explore whether distinct neural populations of a known language network hub —Broca’s area—are specialised for recognising distinct simple word combinations. The phrases consisted of a noun (flag) occurring either with a content word, an adjective (green flag), or with a function word, a determiner (that flag). The key result is that the distribution of neural populations classifying word combination in Broca’s area seems sensitive to neuroanatomical subdivisions within this area, irrespective of task. The information patterns for adjective + noun were localised in its anterior part (BA45) whereas those for determiner + noun were localised in its posterior part (BA44). Our findings provide preliminary answers to the fundamental question of how lexical and grammatical category information interact during simple word combination, with the observation that Broca’s area is sensitive to the recognition of categorical relationships during combinatory processing, based on different demands placed on syntactic and semantic information. This supports the hypothesis that the combinatorial power of language consists of some neural computation capturing phrasal differences when processing linguistic input

Syntax through the looking glass: A review on two-word linguistic processing across behavioral, neuroimaging and neurostimulation studies

In recent years a growing number of studies on syntactic processing has employed basic two-word constructions (e.g., “the tree”) to characterize the fundamental aspects of linguistic composition. This large body of evidence allows, for the first time, to closely examine which cognitive processes and neural substrates support the combination of two syntactic units into a more complex one, mirroring the nature of combinatory operations described in theoretical linguistics. The present review comprehensively examines behavioural, neuroimaging and neurostimulation studies investigating basic syntactic composition, covering more than forty years of psycho- and neuro-linguistic research. Across several paradigms, four key features of syntactic composition have emerged: (1) the rule-based and (2) automatic nature of the combinatorial process, (3) a central role of Broca’s area and the posterior temporal lobe in representing and combining syntactic features, and (4) the reliance on efficient bottom-up integration rather than top-down prediction

The neural correlates of actions: A meta-analytical perspective on motor domains and movement features

Actions can be described in terms of modality of domain (e.g., imitation) and features of the movement (e.g., presence/absence of objects, complex/simple motor outputs). At the neuroanatomical level, conclusive evidence for both modality-specific phenomena and feature processing across multiple action domains remains sparse. Here we ask: (1) What neural resources are required to process actions within specific modalities? (2) Are the same motor features coded through similar neural networks in different modalities? By using quantitative Activation Likelihood Estimation (ALE) and Meta-Analytic Connectivity Modelling (MACM) methods, we obtained cumulative activity distributions of 416 previously published neuroimaging experiments to disentangle functional specificities of six action domains: (1) execution, (2) imitation, (3) observation, (4) imagery, (5) learning, and (6) planning. Our results show distinct functional patterns for the different domains, with cross-modal convergence for execution, imitation and imagery in the posterior Brodmann Area (pBA) 44 of the left inferior frontal gyrus (IFG). Fine-grained analyses in pBA44 reveals that activity in the region does not correlate with movement complexity, but rather with non-object-directed actions. Furthermore, the functional connectivity network seeding in the motor-based localized cluster of pBA44 differs from the connectivity network seeding in the (language-related) anterior BA44. Bringing the results together, we propose that the motor-related network encompassing pBA44 is recruited when processing simple movements, especially when the focus is on finger positioning. This, we believe, questions the role of the left IFG as a domain-general hub for processing syntactic complexity and simultaneously fails to support a mirror neuron hypothesis of action processing

Dissociable contributions of frontal and temporal brain regions to basic semantic composition

Semantic composition is the ability to combine single words to form complex meanings and is an essential component for successful communication. Evidence from neuroimaging studies suggests that semantic composition engages a widely distributed left-hemispheric network, including the anterior temporal lobe, the inferior frontal gyrus and the angular gyrus. To date, the functional relevance of these regions remains unclear. Here, we investigate the impact of lesions to key regions in the semantic network on basic semantic composition. We conducted a multivariate lesion-behaviour mapping study in 36 native German speaking participants with chronic lesions to the language network after left-hemispheric stroke. During the experiment, participants performed a plausibility judgement task on auditorily presented adjective-noun phrases that were either meaningful (‘anxious horse’), anomalous (‘anxious salad’) or had the noun replaced by a pseudoword (‘anxious gufel’), as well as a single-word control condition (‘horse’). We observed that reduced accuracy for anomalous phrases is associated with lesions in left anterior inferior frontal gyrus, whereas increased reaction times for anomalous phrases correlates with lesions in anterior-to-mid temporal lobe. These results indicate that anterior inferior frontal gyrus is relevant for accurate semantic decisions, while anterior-to-mid temporal lobe lesions lead to slowing of the decision for anomalous two-word phrases. These differential effects of lesion location support the notion that anterior inferior frontal gyrus affords executive control for decisions on semantic composition while anterior-to-mid temporal lobe lesions slow the semantic processing of the individual constituents of the phrase

Testing automaticity of syntax using subliminal priming: A behavioural assessment in the German language

Neurophysiological studies investigated the automaticity of syntactic processing under conscious conditions. However, it remains largely unknown whether syntactic analysis proceeds outside of conscious awareness. In this study, we employed a subliminal syntactic priming paradigm to address the automaticity of syntax in minimal phrases. We run four behavioural experiments to test whether recognition of a target’s category (noun or verb) could be influenced by its syntactic relationship with a prime (“a” or “he”). Prime-target relationship was either congruent (“a” + noun, “he” + verb) or incongruent (“a” + verb, “he” + noun). In the pilot and Experiments 1 and 2, verbs included an overt morphological cue (the suffix “-t”), which was absent in Experiment 3. In Experiment 2 we included nonword primes as a baseline condition, to remove processing differences between nouns and verbs independent of syntactic context. Experiments 1 and 2 revealed a subliminal syntactic priming effect: words preceded by syntactically congruent masked primes were processed faster than words preceded by incongruent primes. When no morphological cue was present (Experiment 3), we found no subliminal syntactic priming. Experiment 3 revealed an inhibitory nature of syntactic priming: incongruent primes slowed down the processing of nouns and verbs compared to nonword primes. Our study shows that abstract syntactic representations might be accessed unconsciously in an automatic fashion. Furthermore, the results indicate the role of morphological cues in the early and automatic steps of syntactic analysis

Hierarchical syntactic processing is beyond mere associating: Functional magnetic resonance imaging evidence from a novel artificial grammar

Grammar is central to any natural language. In the past decades, the artificial grammar of the An Bn type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial "Hierarchical syntactic structure-building Grammar" (HG), and compare this to the An Bn grammar as a "Nested associating Grammar" (NG) based on multilevel associations. Thirty-six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build-up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left-hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax-specific processes in the human brain