The neural basis of sign language processing in deaf signers: An activation likelihood estimation meta-analysis

The neurophysiological response during processing of sign language (SL) has been studied since the advent of Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI). Nevertheless, the neural substrates of SL remain subject to debate, especially with regard to involvement and relative lateralization of SL processing without production in (left) inferior frontal gyrus (IFG; e.g., Campbell, MacSweeney, & Waters, 2007; Emmorey, 2006, 2015). Our present contribution is the first to address these questions meta-analytically, by exploring functional convergence on the whole-brain level using previous fMRI and PET studies of SL processing in deaf signers. We screened 163 records in PubMed and Web of Science to identify studies of SL processing in deaf signers conducted with fMRI or PET that reported foci data for one of the two whole-brain contrasts: (1) “SL processing vs. control” or (2) “SL processing vs. low-level baseline”. This resulted in a total of 21 studies reporting 23 experiments matching our selection criteria. We manually extracted foci data and performed a coordinate-based Activation Likelihood Estimation (ALE) analysis using GingerALE (Eickhoff et al., 2009). Our selection criteria and the ALE method allow us to identify regions that are consistently involved in processing SL across studies and tasks. Our analysis reveals that processing of SL stimuli of varying linguistic complexity engages widely distributed bilateral fronto-occipito-temporal networks in deaf signers. We find significant clusters in both hemispheres, with the largest cluster (5240 mm3) being located in left IFG, spanning Broca’s region (posterior BA 45 and the dorsal portion of BA 44). Other clusters are located in right middle and inferior temporal gyrus (BA 37), right IFG (BA 45), left middle occipital gyrus (BA 19), right superior temporal gyrus (BA 22), left precentral and middle frontal gyrus (BA 6 and 8), as well as left insula (BA 13). On these clusters, we calculated lateralization indices using hemispheric and anatomical masks: SL comprehension is slightly left-lateralized globally, and strongly left-lateralized in Broca’s region. Sub-regionally, left-lateralization is strongest in BA 44 (Table 1). Next, we performed a contrast analysis between SL and an independent dataset of action observation in hearing non-signers (Papitto, Friederici, & Zaccarella, 2019) to determine which regions are associated with processing of human actions and movements irrespective of the presence of linguistic information. Only studies of observation of non-linguistic manual actions were included in the final set (n = 26), for example, excluding the handling of objects. Significant clusters involved in the linguistic aspects of SL comprehension were found in left Broca’s region (centered in dorsal BA 44), right superior temporal gyrus (BA 22), and left middle frontal and precentral gyrus (BA 6 and 8; Figure 1A, B, D and E). Meta-analytic connectivity modelling for the surviving cluster in Broca’s region using the BrainMap database then revealed that it is co-activated with the classical language network and functionally primarily associated with cognition and language processing (Figure 1C and D). In line with studies of spoken and written language processing (Zaccarella, Schell, & Friederici, 2017; Friederici, Chomsky, Berwick, Moro, & Bolhuis, 2017), our meta-analysis points to Broca’s region and especially left BA 44 as a hub in the language network that is involved in language processing independent of modality. Right IFG activity is not language-specific but may be specific to the visuo-gestural modality (Campbell et al., 2007). References Amunts, K., Schleicher, A., Bürgel, U., Mohlberg, H., Uylings, H. B., & Zilles, K. (1999). Broca’s region revisited: Cytoarchitecture and intersubject variability. The Journal of Comparative Neurology, 412(2), 319-341. Campbell, R., MacSweeney, M., & Waters, D. (2007). Sign language and the brain: A review. Journal of Deaf Studies and Deaf Education, 13(1), 3-20. doi: 10.1093/deafed/enm035 Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009). Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: A random-effects approach based on empirical estimates of spatial uncertainty. Human Brain Mapping, 30(9), 2907-2926. doi: 10.1002/hbm.20718 Emmorey, K. (2006). The role of Broca’s area in sign language. In Y. Grodzinsky & K. Amunts (Eds.), Broca’s region (p. 169-184). Oxford, England: Oxford UP. Emmorey, K. (2015). The neurobiology of sign language. In A. W. Toga, P. Bandettini, P. Thompson, & K. Friston (Eds.), Brain mapping: An encyclopedic reference (Vol. 3, p. 475-479). London, England: Academic Press. doi: 10.1016/B978-0-12-397025-1.00272-4 Friederici, A. D., Chomsky, N., Berwick, R. C., Moro, A., & Bolhuis, J. J. (2017). Language, mind and brain. Nature Human Behaviour. doi: 10.1038/s41562-017-0184-4 Matsuo, K., Chen, S.-H. A., & Tseng, W.-Y. I. (2012). AveLI: A robust lateralization index in functional magnetic resonance imaging using unbiased threshold-free computation. Journal of Neuroscience Methods, 205(1), 119-129. doi: 10.1016/j.jneumeth.2011.12.020 Papitto, G., Friederici, A. D., & Zaccarella, E. (2019). A neuroanatomical comparison of action domains using Activation Likelihood Estimation meta-analysis [Unpublished Manuscript, Max Planck Institute for Human Cognitive & Brain Sciences]. Leipzig, Germany. Zaccarella, E., Schell, M., & Friederici, A. D. (2017). Reviewing the functional basis of the syntactic Merge mechanism for language: A coordinate-based activation likelihood estimation meta-analysis. Neuroscience & Biobehavioral Reviews, 80, 646-656. doi: 10.1016/j.neubiorev.2017.06.01

Syntactic and semantic contributions of pitch accents during sentence comprehension

Syntactic, semantic, and prosodic cues all establish expectations that guide sentence comprehension. In the prosodic domain, pitch accents can assign contrastive focus and resolve a syntactically ambiguous phrase. However, can prosodic focus marking (by pitch accenting) influence the interpretation of a sentence in the presence of syntactic and semantic cues? Our auditory experiment revolved around the sentence (in German) “Yesterday the policeman arrested the thief, not the murderer”. A pitch accent on either POLICEMAN or THIEF placed one of those arguments in contrastive focus with the ellipsis structure ("the murderer”). The two contrasted arguments could contain violations: in the syntax condition, the grammatical case of the article in the ellipsis structure mismatched the focused constituent in the main clause (nominative vs. accusative). In the semantic condition, the thematic roles of the contrasted words were incongruent (typical agent vs. patient roles of “arrest”). Visual comprehension questions probed the agent/patient role of the arguments in the sentence (subject or object), followed by a button-press response. Reaction times showed that if the pitch accent marked syntactic information that mismatched the syntactic information in the ellipsis structure, responses were delayed. The direction of the semantic effect depended on the focused noun. The response patterns showed that participants were led by the syntactic information to make their syntactic judgements, despite a conflicting expectation established by prosody. The experiment shows that pitch accents establish a syntactic expectation during sentence comprehension. However, these expectations are overwritten by incoming syntactic information to yield an interpretation of the sentence

Bach speaks: A cortical "language-network" serves the processing of music

The aim of the present study was the investigation of neural correlates of music processing with fMRI. Chord sequences were presented to the participants, infrequently containing unexpected musical events. These events activated the areas of Broca and Wernicke, the superior temporal sulcus, Heschl's gyrus, both planum polare and planum temporale, as well as the anterior superior insular cortices. Some of these brain structures have previously been shown to be involved in music processing, but the cortical network comprising all these structures has up to now been thought to be domain-specific for language processing. To what extent this network might also be activated by the processing of non-linguistic information has remained unknown. The present fMRI-data reveal that the human brain employs this neuronal network also for the processing of musical information, suggesting that the cortical network known to support language processing is less domain-specific than previously believed

Language and action in Broca’s area: Computational differentiation and cortical segregation

Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca’s area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca’s subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world

The neural correlates of actions: A meta-analytical perspective on motor domains and movement features

Actions can be described in terms of modality of domain (e.g., imitation) and features of the movement (e.g., presence/absence of objects, complex/simple motor outputs). At the neuroanatomical level, conclusive evidence for both modality-specific phenomena and feature processing across multiple action domains remains sparse. Here we ask: (1) What neural resources are required to process actions within specific modalities? (2) Are the same motor features coded through similar neural networks in different modalities? By using quantitative Activation Likelihood Estimation (ALE) and Meta-Analytic Connectivity Modelling (MACM) methods, we obtained cumulative activity distributions of 416 previously published neuroimaging experiments to disentangle functional specificities of six action domains: (1) execution, (2) imitation, (3) observation, (4) imagery, (5) learning, and (6) planning. Our results show distinct functional patterns for the different domains, with cross-modal convergence for execution, imitation and imagery in the posterior Brodmann Area (pBA) 44 of the left inferior frontal gyrus (IFG). Fine-grained analyses in pBA44 reveals that activity in the region does not correlate with movement complexity, but rather with non-object-directed actions. Furthermore, the functional connectivity network seeding in the motor-based localized cluster of pBA44 differs from the connectivity network seeding in the (language-related) anterior BA44. Bringing the results together, we propose that the motor-related network encompassing pBA44 is recruited when processing simple movements, especially when the focus is on finger positioning. This, we believe, questions the role of the left IFG as a domain-general hub for processing syntactic complexity and simultaneously fails to support a mirror neuron hypothesis of action processing

Left posterior inferior frontal gyrus is causally involved in complex sentence comprehension

INTRODUCTION Storage and reordering of words are two core processes required for successful sentence comprehension. Storage is necessary whenever the verb and its arguments (i.e., subject and object) are separated over a long distance, while reordering is necessary whenever the argument order is atypical (e.g., object-first order in German, where subject-first order is typical). Previous neuroimaging work (Meyer et al., 2012) has associated storage with the left planum temporale (PT), and reordering with the left posterior inferior frontal gyrus (pIFG). However, it is unclear whether left PT and pIFG are indeed causally relevant for storage and reordering, respectively. Here, we tested the necessity of the PT and pIFG for storage and reordering using repetitive transcranial magnetic stimulation (rTMS). METHODS We applied either effective online rTMS (5 pulses at 10 Hz) over PT or pIFG, or sham rTMS, while subjects listened to sentences that independently manipulated storage demands (short vs. long argument–verb distance) and reordering demands (subject– vs. object-first argument order). We employed behavioral modeling, using a drift diffusion model, to assess rTMS-induced disruption of sentence comprehension. RESULTS We found that rTMS over pIFG, but not PT, selectively impaired reordering during the processing of sentences with a long argument–verb distance. Specifically, relative to sham rTMS, rTMS over pIFG significantly increased the performance decline for object– vs. subject-first long-distance sentences (t23 = 2.86; p = 0.009). This effect was anatomically specific as the same comparison for PT stimulation was far from significant (t23 = −0.11; p = 0.9), and a direct across-sites comparison showed that the pIFG effect was significantly stronger (t23 = −2.62; p = 0.015). CONCLUSION Our results provide the first causal evidence that the left pIFG supports the reordering of arguments in long-distance sentences. We thereby substantially extend previous neuroimaging studies that showed a correlation between pIFG activation and reordering demands. Together with previous evidence (Lauro et al., 2010), our findings indicate that the left pIFG crucially supports the comprehension of syntactically complex sentences. These results might extend to other domains, such as music (Maess et al., 2001) and action (Clerget et al., 2009), indicating a domain-general role of left pIFG in the processing of hierarchically-structured sequences

Dissociable contributions of frontal and temporal brain regions to basic semantic composition

Semantic composition is the ability to combine single words to form complex meanings and is an essential component for successful communication. Evidence from neuroimaging studies suggests that semantic composition engages a widely distributed left-hemispheric network, including the anterior temporal lobe, the inferior frontal gyrus and the angular gyrus. To date, the functional relevance of these regions remains unclear. Here, we investigate the impact of lesions to key regions in the semantic network on basic semantic composition. We conducted a multivariate lesion-behaviour mapping study in 36 native German speaking participants with chronic lesions to the language network after left-hemispheric stroke. During the experiment, participants performed a plausibility judgement task on auditorily presented adjective-noun phrases that were either meaningful (‘anxious horse’), anomalous (‘anxious salad’) or had the noun replaced by a pseudoword (‘anxious gufel’), as well as a single-word control condition (‘horse’). We observed that reduced accuracy for anomalous phrases is associated with lesions in left anterior inferior frontal gyrus, whereas increased reaction times for anomalous phrases correlates with lesions in anterior-to-mid temporal lobe. These results indicate that anterior inferior frontal gyrus is relevant for accurate semantic decisions, while anterior-to-mid temporal lobe lesions lead to slowing of the decision for anomalous two-word phrases. These differential effects of lesion location support the notion that anterior inferior frontal gyrus affords executive control for decisions on semantic composition while anterior-to-mid temporal lobe lesions slow the semantic processing of the individual constituents of the phrase

‘Mining the materials’: A framework for student-led self-study task creation

Meaningful independent learning is rightly viewed as a central component of successful study in L2. Given that the considerable majority of learners’ time is spent outside the classroom, the self-study space has become a source of great intrigue for English language teachers (Benson and Reinders, 2011). However, precisely because self-directed learning lies beyond the typical boundaries of the teacher’s gaze, it is influenced by a variety of factors, not least learners’ familiarity with effective independent learning practices. This summary article traces and evaluates the implementation of a framework for student-led self-study task creation with a group of 14 foundation pre-sessional students making the transition from secondary to tertiary study at the University of Glasgow with little or no existing concept of effective self-study practices. The trial aimed to provide a space for students to evaluate their strengths and weaknesses in English and establish independent learning priorities, as well as a more critical awareness (‘mining’) of regular classroom tasks as potential models for independent learning activities. Obtaining feedback at regular intervals, coupled with data from weekly reflection cycles, the investigation tracked developments in self-study practices while highlighting obstacles to enhanced independent learning. The trial also presented plentiful opportunities to reflect on the definition of effectiveness with regard to independent learning

Longitudinal trajectories of electrophysiological mismatch responses in infant speech discrimination differ across speech features

Infants rapidly advance in their speech perception, electrophysiologically reflected in the transition from an immature, positive-going to an adult-like, negative-going mismatch response (MMR) to auditory deviancy. Although the MMR is a common tool to study speech perception development, it is not yet completely understood how different speech contrasts affect the MMR’s characteristics across development. Thus, a systematic longitudinal investigation of the MMR’s maturation depending on speech contrast is necessary. We here longitudinally explored the maturation of the infant MMR to four critical speech contrasts: consonant, vowel, vowel-length, and pitch. MMRs were obtained when infants (n = 58) were 2, 6 and 10 months old. To evaluate the maturational trajectory of MMRs, we applied second-order latent growth curve models. Results showed positive-going MMR amplitudes to all speech contrasts across all assessment points that decreased over time towards an adult-like negativity. Notably, the developmental trajectories of speech contrasts differed, implying that infant speech perception matures with different rates and trajectories throughout the first year, depending on the studied auditory feature. Our results suggest that stimulus-dependent maturational trajectories need to be considered when drawing conclusions about infant speech perception development reflected by the infant MMR

Pitch accents create dissociable syntactic and semantic expectations during sentence processing

The language system uses syntactic, semantic, as well as prosodic cues to efficiently guide auditory sentence comprehension. Prosodic cues, such as pitch accents, can build expectations about upcoming sentence elements. This study investigates to what extent syntactic and semantic expectations generated by pitch accents can be dissociated and if so, which cues take precedence when contradictory information is present. We used sentences in which one out of two nominal constituents was placed in contrastive focus with a third one. All noun phrases carried overt syntactic information (case-marking of the determiner) and semantic information (typicality of the thematic role of the noun). Two experiments (a sentence comprehension and a sentence completion task) show that focus, marked by pitch accents, established expectations in both syntactic and semantic domains. However, only the syntactic expectations, when violated, were strong enough to interfere with sentence comprehension. Furthermore, when contradictory cues occurred in the same sentence, the local syntactic cue (case-marking) took precedence over the semantic cue (thematic role), and overwrote previous information cued by prosody. The findings indicate that during auditory sentence comprehension the processing system integrates different sources of information for argument role assignment, yet primarily relies on syntactic information