Graded Visual Attention Modulates Brain Responses Evoked by Task-irrelevant Auditory Pitch Changes

Previous studies suggested that auditory change-specific neural responses are attention-independent and reflect central auditory processing. The automaticity of the brain's response to infrequent changes in pitch within a series of auditory tone pips was examined in parallel functional magnetic resonance imaging (fMRI) and event-related potential (ERP) studies. Subjects performed a continuous perceptual-motor visual tracking task at two levels of difficulty while simultaneously hearing a series of task-irrelevant standard tone pips and infrequent pitch-deviant tones. fMRI results revealed that the unattended pitch-deviant tones strongly activated superior temporal and frontal cortical regions. These activations were significantly modulated by the tracking difficulty of the primary task. ERP results revealed that the amplitude of the scalp-negative component evoked by deviant tones (MMN) was attenuated during the more difficult tracking task. Our results demonstrate that the brain's response to task-irrelevant sensory changes is strongly influenced by intermodal attentional demands

Task relevance modulates the behavioural and neural effects of sensory predictions

The brain is thought to generate internal predictions to optimize behaviour. However, it is unclear whether predictions signalling is an automatic brain function or depends on task demands. Here, we manipulated the spatial/temporal predictability of visual targets, and the relevance of spatial/temporal information provided by auditory cues. We used magnetoencephalography (MEG) to measure participants' brain activity during task performance. Task relevance modulated the influence of predictions on behaviour: spatial/temporal predictability improved spatial/temporal discrimination accuracy, but not vice versa. To explain these effects, we used behavioural responses to estimate subjective predictions under an ideal-observer model. Model-based time-series of predictions and prediction errors (PEs) were associated with dissociable neural responses: predictions correlated with cue-induced beta-band activity in auditory regions and alpha-band activity in visual regions, while stimulus-bound PEs correlated with gamma-band activity in posterior regions. Crucially, task relevance modulated these spectral correlates, suggesting that current goals influence PE and prediction signalling

The impact of reward value on early sensory processing and its interaction with selective attention

Reward value affects the earliest stages of sensory perception. Whereas a host of previous studies have investigated the underlying mechanisms of reward-driven modulation of visual perception, reward effects in other sensory modalities have remained underexplored. Specifically, it has remained unknown how reward signals should be coordinated and communicated across sensory modalities. The current PhD thesis aimed to gain insight into the underlying mechanisms of reward-driven modulation of perception and its interaction with attention across sensory modalities. To this end, three experiments were conducted to identify the behavioral and electrophysiological correlates of reward effects. In Study 1, we found that high reward, task-irrelevant visual cues (intra-modal) elicited an early suppression of visual event-related potentials (ERPs). High reward auditory cues (cross-modal), on the other hand, led to a later modulation of visual ERPs and facilitated behavioral performance. Study 2 tested the dependence of reward effects on the spatial and temporal arrangement of intra-modal and cross-modal cues relative to the target, and showed that each reward cue maximally exerts its effect under a specific size of attentional focus. Study 3 explicitly manipulated the spatial attention and tested how reward associations of an audiovisual stimulus influence the allocation of attention. We found that auditory rewards enhanced the attentional modulation of both visual and auditory ERPs. Interestingly, although visual rewards did not lead to a distinguishable ERP modulation, they led to strong modulations when they were combined with auditory rewards, suggesting that integration across modalities boosts the reward effects. Taken together, the current PhD thesis identified the behavioral and neural signatures of reward-driven modulation of perception under different modes of reward signaling and different degrees of attentional engagement. Our findings inspire a two-stage model of reward processing, with local, intra-modal reward effects occurring at an early stage and long-range, multimodal reward effects arising at a later stage. Cross-modal reward signals have important ramifications for clinical applications where the impaired function of one sense can be rehabilitated by motivational signals conveyed through another sensory modality.2021-11-2

Taking attention away from the auditory modality : investigations of the effect on speech processing using machine learning

Real-world speech processing often takes place in complex multisensory environments. Listeners may need to prioritize sensory inputs from modalities other than audition. Selective attention is thought to be critical in selecting the sensory modality most relevant to the task at hand. Two critical research questions have driven crossmodal attention research thus far: first, how early does crossmodal attention influences processing in the unattended modality? Second, is there a limitation in attentional resources between sensory modalities? Set within the context of this prior work, this dissertation aims to examine the effects of crossmodal attention on speech processing when sensory inputs from vision are prioritized. In study 1, we demonstrate that modulating visual perceptual load can impact the early sensory representation of linguistically-relevant pitch contours (Mandarin tones), a suprasegmental feature that is critical to the percept of lexical tones. Further, we provide novel evidence that the impact of the visual load is highly dependent on the predictability of the incoming speech stream. In study 2, we utilized ecologically valid, continuous speech and tested the extent to which dividing attention to a visual task affects neural processing of speech signals. We show that dividing attention between auditory and visual tasks leads to both behavioral and electrophysiological costs in the processing of continuous speech stimuli. The results also demonstrate that the neural encoding of suprasegmental features (e.g., envelope and fundamental frequency) in continuous speech is modulated by diverting attention away from the auditory modality. In contrast, the neural encoding of segmental features (e.g., phonetic features) may be unaffected by taking attention away from the auditory stream. The theoretical and practical implications of the two studies are discussed.Communication Sciences and Disorder

Visual attentional load influences plasticity in the human motor cortex

Neural plasticity plays a critical role in learning, memory, and recovery from injury to the nervous system. Although much is known about the physical and physiological determinants of plasticity, little is known about the influence of cognitive factors. In this study, we investigated whether selective attention plays a role in modifying changes in neural excitability reflecting long-term potentiation (LTP)like plasticity. We induced LTP-like effects in the hand area of the human motor cortex using transcranial magnetic stimulation (TMS). During the induction of plasticity, participants engaged in a visual detection task with either low or high attentional demands. Changes in neural excitability were assessed by measuring motor-evoked potentials in a small hand muscle before and after the TMS procedures. In separate experiments plasticity was induced either by paired associative stimulation (PAS) or intermittent theta-burst stimulation (iTBS). Because these procedures induce different forms of LTP-like effects, they allowed us to investigate the generality of any attentional influence on plasticity. In both experiments reliable changes in motor cortex excitability were evident under low-load conditions, but this effect was eliminated under high-attentional load. In a third experiment we investigated whether the attentional task was associated with ongoing changes in the excitability of motor cortex, but found no difference in evoked potentials across the levels of attentional load. Our findings indicate that in addition to their role in modifying sensory processing, mechanisms of attention can also be a potent modulator of cortical plasticity

Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition

In order to understand the working brain as a network, it is essential to identify the mechanisms by which information is gated between regions. We here propose that information is gated by inhibiting task-irrelevant regions, thus routing information to task-relevant regions. The functional inhibition is reflected in oscillatory activity in the alpha band (8–13 Hz). From a physiological perspective the alpha activity provides pulsed inhibition reducing the processing capabilities of a given area. Active processing in the engaged areas is reflected by neuronal synchronization in the gamma band (30–100 Hz) accompanied by an alpha band decrease. According to this framework the brain could be studied as a network by investigating cross-frequency interactions between gamma and alpha activity. Specifically the framework predicts that optimal task performance will correlate with alpha activity in task-irrelevant areas. In this review we will discuss the empirical support for this framework. Given that alpha activity is by far the strongest signal recorded by EEG and MEG, we propose that a major part of the electrophysiological activity detected from the working brain reflects gating by inhibition

Perceptual Load-Dependent Neural Correlates of Distractor Interference Inhibition

The load theory of selective attention hypothesizes that distractor interference is suppressed after perceptual processing (i.e., in the later stage of central processing) at low perceptual load of the central task, but in the early stage of perceptual processing at high perceptual load. Consistently, studies on the neural correlates of attention have found a smaller distractor-related activation in the sensory cortex at high relative to low perceptual load. However, it is not clear whether the distractor-related activation in brain regions linked to later stages of central processing (e.g., in the frontostriatal circuits) is also smaller at high rather than low perceptual load, as might be predicted based on the load theory.We studied 24 healthy participants using functional magnetic resonance imaging (fMRI) during a visual target identification task with two perceptual loads (low vs. high). Participants showed distractor-related increases in activation in the midbrain, striatum, occipital and medial and lateral prefrontal cortices at low load, but distractor-related decreases in activation in the midbrain ventral tegmental area and substantia nigra (VTA/SN), striatum, thalamus, and extensive sensory cortices at high load.Multiple levels of central processing involving midbrain and frontostriatal circuits participate in suppressing distractor interference at either low or high perceptual load. For suppressing distractor interference, the processing of sensory inputs in both early and late stages of central processing are enhanced at low load but inhibited at high load

Neural dynamics of selective attention to speech in noise

This thesis investigates how the neural system instantiates selective attention to speech in challenging acoustic conditions, such as spectral degradation and the presence of background noise. Four studies using behavioural measures, magneto- and electroencephalography (M/EEG) recordings were conducted in younger (20–30 years) and older participants (60–80 years). The overall results can be summarized as follows. An EEG experiment demonstrated that slow negative potentials reflect participants’ enhanced allocation of attention when they are faced with more degraded acoustics. This basic mechanism of attention allocation was preserved at an older age. A follow-up experiment in younger listeners indicated that attention allocation can be further enhanced in a context of increased task-relevance through monetary incentives. A subsequent study focused on brain oscillatory dynamics in a demanding speech comprehension task. The power of neural alpha oscillations (~10 Hz) reflected a decrease in demands on attention with increasing acoustic detail and critically also with increasing predictiveness of the upcoming speech content. Older listeners’ behavioural responses and alpha power dynamics were stronger affected by acoustic detail compared with younger listeners, indicating that selective attention at an older age is particularly dependent on the sensory input signal. An additional analysis of listeners’ neural phase-locking to the temporal envelopes of attended speech and unattended background speech revealed that younger and older listeners show a similar segregation of attended and unattended speech on a neural level. A dichotic listening experiment in the MEG aimed at investigating how neural alpha oscillations support selective attention to speech. Lateralized alpha power modulations in parietal and auditory cortex regions predicted listeners’ focus of attention (i.e., left vs right). This suggests that alpha oscillations implement an attentional filter mechanism to enhance the signal and to suppress noise. A final behavioural study asked whether acoustic and semantic aspects of task-irrelevant speech determine how much it interferes with attention to task-relevant speech. Results demonstrated that younger and older adults were more distracted when acoustic detail of irrelevant speech was enhanced, whereas predictiveness of irrelevant speech had no effect. All findings of this thesis are integrated in an initial framework for the role of attention for speech comprehension under demanding acoustic conditions

Neural correlates of audiovisual motion capture

Visual motion can affect the perceived direction of auditory motion (i.e., audiovisual motion capture). It is debated, though, whether this effect occurs at perceptual or decisional stages. Here, we examined the neural consequences of audiovisual motion capture using the mismatch negativity (MMN), an event-related brain potential reflecting pre-attentive auditory deviance detection. In an auditory-only condition occasional changes in the direction of a moving sound (deviant) elicited an MMN starting around 150 ms. In an audiovisual condition, auditory standards and deviants were synchronized with a visual stimulus that moved in the same direction as the auditory standards. These audiovisual deviants did not evoke an MMN, indicating that visual motion reduced the perceptual difference between sound motion of standards and deviants. The inhibition of the MMN by visual motion provides evidence that auditory and visual motion signals are integrated at early sensory processing stages

Surprise responses in the human brain demonstrate statistical learning under high concurrent cognitive demand

The ability to learn about regularities in the environment and to make predictions about future events is fundamental for adaptive behaviour. We have previously shown that people can implicitly encode statistical regularities and detect violations therein, as reflected in neuronal responses to unpredictable events that carry a unique prediction error signature. In the real world, however, learning about regularities will often occur in the context of competing cognitive demands. Here we asked whether learning of statistical regularities is modulated by concurrent cognitive load. We compared electroencephalographic metrics associated with responses to pure-tone sounds with frequencies sampled from narrow or wide Gaussian distributions. We showed that outliers evoked a larger response than those in the centre of the stimulus distribution (i.e., an effect of surprise) and that this difference was greater for physically identical outliers in the narrow than in the broad distribution. These results demonstrate an early neurophysiological marker of the brain’s ability to implicitly encode complex statistical structure in the environment. Moreover, we manipulated concurrent cognitive load by having participants perform a visual working memory task while listening to these streams of sounds. We again observed greater prediction error responses in the narrower distribution under both low and high cognitive load. Furthermore, there was no reliable reduction in prediction error magnitude under high-relative to low cognitive load. Our findings suggest that statistical learning is not a capacity limited process, and that it proceeds automatically even when cognitive resources are taxed by concurrent demands