Seeing a talking face matters: The relationship between cortical tracking of continuous auditory ‐visual speech and gaze behaviour in infants, children and adults

Available online 15 April 2022An auditory-visual speech benefit, the benefit that visual speech cues bring to auditory speech perception, is experienced from early on in infancy and continues to be experienced to an increasing degree with age. While there is both behavioural and neurophysiological evidence for children and adults, only behavioural evidence exists for infants –as no neurophysiological study has provided a comprehensive examination of the auditory- visual speech benefit in infants. It is also surprising that most studies on auditory-visual speech benefit do not concurrently report looking behaviour especially since the auditory-visual speech benefit rests on the assumption that listeners attend to a speaker’s talking face and that there are meaningful individual differences in looking behaviour. To address these gaps, we simultaneously recorded electroencephalographic (EEG) and eye-tracking data of 5-month-olds, 4-year-olds and adults as they were presented with a speaker in auditory-only (AO), visual- only (VO), and auditory-visual (AV) modes. Cortical tracking analyses that involved forward encoding models of the speech envelope revealed that there was an auditory-visual speech benefit [i.e., AV > ( A + V )], evident in 5-month-olds and adults but not 4-year-olds. Examination of cortical tracking accuracy in relation to looking behaviour, showed that infants’ relative attention to the speaker’s mouth (vs. eyes) was positively correlated with cortical tracking accuracy of VO speech, whereas adults’ attention to the display overall was negatively correlated with cortical tracking accuracy of VO speech. This study provides the first neurophysiological evidence of auditory-visual speech benefit in infants and our results suggest ways in which current models of speech processing can be fine-tuned.This research was funded by a doctoral scholarship to the first author funded by the MARCS Institute at Western Sydney University and the HEARing Cooperative Research Centre (CRC), and by HEARingCRC funding to the last author. The second author’s work is supported by the Basque Government through the BERC 2018–2021 program, and PIBA PI-2019–0054, and by the Spanish Ministry of Science and Innovation through the Ramon y Cajal Research Fellowship, PID2019– 105528GA-I00

INVESTIGATION ON NEURAL RESPONSES RELATED TO THE LOCALIZATION OF NATURAL SOUNDS

Spatial hearing allows the localization of sounds in complex acoustic environments. There is considerable evidence that this neural system rapidly adapts to changes in sensory inputs and behavioral goals. However, the mechanisms underlying this context-dependent coding are not well understood. In fact, previous studies on sound localization have mainly focused on the perception of simple artificial sounds, such as white-noise or pure tone bursts. In addition, previous research has generally investigated the localization of sounds in the frontal hemicircle while ignoring rear sources. However, their localization is evolutionary relevant and may show different neural coding, given the inherent lack of visual information. Here we present a pilot electroencephalography (EEG) study to identify robust indices of sound localization from participants listening to a short natural sound from eight source positions on the horizontal plane. We discuss a procedure to perform a within-subject classification of the perceived sound direction. Preliminary results suggest a pool of discriminative subject-specific temporal and topographical features correlated with the characteristics of the acoustic event. Our preliminary analysis has identified temporal and topographical features that are sensitive to spatial localization, leading to significant decoding of sounds direction for individual subjects. This pilot study adds to the literature a methodological approach that will lead to the objective classification of natural sounds location from EEG responses

Dissociable electrophysiological measures of natural language processing reveal differences in speech comprehension strategy in healthy ageing

Healthy ageing leads to changes in the brain that impact upon sensory and cognitive processing. It is not fully clear how these changes affect the processing of everyday spoken language. Prediction is thought to play an important role in language comprehension, where information about upcoming words is pre-activated across multiple representational levels. However, evidence from electrophysiology suggests differences in how older and younger adults use context-based predictions, particularly at the level of semantic representation. We investigate these differences during natural speech comprehension by presenting older and younger subjects with continuous, narrative speech while recording their electroencephalogram. We use time-lagged linear regression to test how distinct computational measures of (1) semantic dissimilarity and (2) lexical surprisal are processed in the brains of both groups. Our results reveal dissociable neural correlates of these two measures that suggest differences in how younger and older adults successfully comprehend speech. Specifically, our results suggest that, while younger and older subjects both employ context-based lexical predictions, older subjects are significantly less likely to pre-activate the semantic features relating to upcoming words. Furthermore, across our group of older adults, we show that the weaker the neural signature of this semantic pre-activation mechanism, the lower a subject's semantic verbal fluency score. We interpret these findings as prediction playing a generally reduced role at a semantic level in the brains of older listeners during speech comprehension and that these changes may be part of an overall strategy to successfully comprehend speech with reduced cognitive resources

Neurodegenerative phagocytes mediate synaptic stripping in Neuro-HIV.

Glial cell activation is a hallmark of several neurodegenerative and neuroinflammatory diseases. During HIV infection, neuroinflammation is associated with cognitive impairment, even during sustained long-term suppressive antiretroviral therapy. However, the cellular subsets contributing to neuronal damage in the CNS during HIV infection remain unclear. Using post-mortem brain samples from eight HIV patients and eight non-neurological disease controls, we identify a subset of CNS phagocytes highly enriched in LGALS3, CTSB, GPNMB and HLA-DR, a signature identified in the context of ageing and neurodegeneration. In HIV patients, the presence of this phagocyte phenotype was associated with synaptic stripping, suggesting an involvement in the pathogenesis of HIV-associated neurocognitive disorder. Taken together, our findings elucidate some of the molecular signatures adopted by CNS phagocytes in HIV-positive patients and contribute to the understanding of how HIV might pave the way to other forms of cognitive decline in ageing HIV patient populations

Decoding the auditory brain with canonical component analysis

The relation between a stimulus and the evoked brain response can shed light on perceptual processes within the brain. Signals derived from this relation can also be harnessed to control external devices for Brain Computer Interface (BCI) applications. While the classic event-related potential (ERP) is appropriate for isolated stimuli, more sophisticated “decoding” strategies are needed to address continuous stimuli such as speech, music or environmental sounds. Here we describe an approach based on Canonical Correlation Analysis (CCA) that finds the optimal transform to apply to both the stimulus and the response to reveal correlations between the two. Compared to prior methods based on forward or backward models for stimulus-response mapping, CCA finds significantly higher correlation scores, thus providing increased sensitivity to relatively small effects, and supports classifier schemes that yield higher classification scores. CCA strips the brain response of variance unrelated to the stimulus, and the stimulus representation of variance that does not affect the response, and thus improves observations of the relation between stimulus and response

Emergence of the cortical encoding of phonetic features in the first year of life.

Even prior to producing their first words, infants are developing a sophisticated speech processing system, with robust word recognition present by 4-6 months of age. These emergent linguistic skills, observed with behavioural investigations, are likely to rely on increasingly sophisticated neural underpinnings. The infant brain is known to robustly track the speech envelope, however previous cortical tracking studies were unable to demonstrate the presence of phonetic feature encoding. Here we utilise temporal response functions computed from electrophysiological responses to nursery rhymes to investigate the cortical encoding of phonetic features in a longitudinal cohort of infants when aged 4, 7 and 11 months, as well as adults. The analyses reveal an increasingly detailed and acoustically invariant phonetic encoding emerging over the first year of life, providing neurophysiological evidence that the pre-verbal human cortex learns phonetic categories. By contrast, we found no credible evidence for age-related increases in cortical tracking of the acoustic spectrogram

Childhood tuberculosis is associated with decreased abundance of T cell gene transcripts and impaired T cell function

The WHO estimates around a million children contract tuberculosis (TB) annually with over 80 000 deaths from dissemination of infection outside of the lungs. The insidious onset and association with skin test anergy suggests failure of the immune system to both recognise and respond to infection. To understand the immune mechanisms, we studied genome-wide whole blood RNA expression in children with TB meningitis (TBM). Findings were validated in a second cohort of children with TBM and pulmonary TB (PTB), and functional T-cell responses studied in a third cohort of children with TBM, other extrapulmonary TB (EPTB) and PTB. The predominant RNA transcriptional response in children with TBM was decreased abundance of multiple genes, with 140/204 (68%) of all differentially regulated genes showing reduced abundance compared to healthy controls. Findings were validated in a second cohort with concordance of the direction of differential expression in both TBM (r2 = 0.78 p = 2x10-16) and PTB patients (r2 = 0.71 p = 2x10-16) when compared to a second group of healthy controls. Although the direction of expression of these significant genes was similar in the PTB patients, the magnitude of differential transcript abundance was less in PTB than in TBM. The majority of genes were involved in activation of leucocytes (p = 2.67E-11) and T-cell receptor signalling (p = 6.56E-07). Less abundant gene expression in immune cells was associated with a functional defect in T-cell proliferation that recovered after full TB treatment (p<0.0003). Multiple genes involved in T-cell activation show decreased abundance in children with acute TB, who also have impaired functional T-cell responses. Our data suggest that childhood TB is associated with an acquired immune defect, potentially resulting in failure to contain the pathogen. Elucidation of the mechanism causing the immune paresis may identify new treatment and prevention strategies

A plea for equitable global access to COVID‐19 diagnostics, vaccination and therapy: The NeuroCOVID‐19 Task Force of the European Academy of Neurology

Coronavirus disease 2019 (COVID‐19), a multi‐organ disease caused by severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), continues to challenge health and care systems around the globe. The pandemic has disrupted acute neurology services and routine patient care and has impacted the clinical course in patients with chronic neurological disease. COVID‐19 appears to have exposed inequalities of societies and healthcare systems and had a disproportionate impact on already vulnerable communities. The next challenge will be to set up initiatives to stop disparities in all aspects related to COVID‐19. From the medical perspective, there is a need to consider inequalities in prevention, treatment and long‐term consequences. Some of the issues of direct relevance to neurologists are summarised. With this appraisal, the European Academy of Neurology NeuroCOVID‐19 Task Force intends to raise awareness of the potential impact of COVID‐19 on inequalities in healthcare and calls for action to prevent disparity at individual, national and supranational levels