Audio-visual synchrony and feature-selective attention co-amplify early visual processing

Our brain relies on neural mechanisms of selective attention and converging sensory processing to efficiently cope with rich and unceasing multisensory inputs. One prominent assumption holds that audio-visual synchrony can act as a strong attractor for spatial attention. Here, we tested for a similar effect of audio-visual synchrony on feature-selective attention. We presented two superimposed Gabor patches that differed in colour and orientation. On each trial, participants were cued to selectively attend to one of the two patches. Over time, spatial frequencies of both patches varied sinusoidally at distinct rates (3.14 and 3.63 Hz), giving rise to pulse-like percepts. A simultaneously presented pure tone carried a frequency modulation at the pulse rate of one of the two visual stimuli to introduce audio-visual synchrony. Pulsed stimulation elicited distinct time-locked oscillatory electrophysiological brain responses. These steady-state responses were quantified in the spectral domain to examine individual stimulus processing under conditions of synchronous versus asynchronous tone presentation and when respective stimuli were attended versus unattended. We found that both, attending to the colour of a stimulus and its synchrony with the tone, enhanced its processing. Moreover, both gain effects combined linearly for attended in-sync stimuli. Our results suggest that audio-visual synchrony can attract attention to specific stimulus features when stimuli overlap in space

A crossmodal crossover: opposite effects of visual and auditory perceptual load on steady-state evoked potentials to irrelevant visual stimuli

Mechanisms of attention are required to prioritise goal-relevant sensory events under conditions of stimulus competition. According to the perceptual load model of attention, the extent to which task-irrelevant inputs are processed is determined by the relative demands of discriminating the target: the more perceptually demanding the target task, the less unattended stimuli will be processed. Although much evidence supports the perceptual load model for competing stimuli within a single sensory modality, the effects of perceptual load in one modality on distractor processing in another is less clear. Here we used steady-state evoked potentials (SSEPs) to measure neural responses to irrelevant visual checkerboard stimuli while participants performed either a visual or auditory task that varied in perceptual load. Consistent with perceptual load theory, increasing visual task load suppressed SSEPs to the ignored visual checkerboards. In contrast, increasing auditory task load enhanced SSEPs to the ignored visual checkerboards. This enhanced neural response to irrelevant visual stimuli under auditory load suggests that exhausting capacity within one modality selectively compromises inhibitory processes required for filtering stimuli in another

Investigating how neural entrainment relates to beat perception by disentangling the stimulus-driven response

Beat perception – the ability to perceive a steady pulse in music – is nearly ubiquitous in humans, but the neural mechanisms underlying this ability are unknown. A growing number of electroencephalography (EEG) studies suggest that beat perception is related to neural entrainment, a phenomenon in which cyclic changes in the excitability of populations of neurons synchronize with a rhythmic stimulus. However, the relationship between acoustically-driven and entrainment-driven neural activity is unclear. This thesis presents EEG research that extends our understanding neural entrainment is related to beat perception by characterizing, equating, and finally removing the stimulus-driven response in the neural signal isolating the entrainment-driven responses. Chapter 1 presents a general overview of how neural entrainment may relate to beat perception, the common methods of measuring neural entrainment, and current debates in the literature about how best to account for the stimulus-driven response in the neural signal and also what the neural power spectrum reflects. Chapter 2 presents research on how perceptual and acoustic factors in auditory stimuli influence neural spectral power in a series of experiments in which beat strength, tone duration, and onset/offset ramp duration were manipulated. The results suggest that both perceptual and acoustic factors influence neural spectral power, and that accounting for the stimulus-driven response in the neural spectrum is more complicated than previously assumed. Chapter 3 presents research on how power and phase of the neural signal relate to beat strength and beat location while controlling the stimulus-driven response. The results indicated a relationship between neural entrainment and beat strength, and also, between oscillatory phase and beat location. Chapter 4 presents research on the potential neural mechanisms of beat perception by examining neural activity during a silent immediately after rhythm perception for testing for ongoing, oscillatory activity. The results, although not statistically robust, suggest that entrained activity continues into silence, indicating a relationship between neural entrainment and beat perception. Chapter 5 presents a general discussion of Chapters 2-4 in the context of the existing literature, limitations, and broader interpretations of how these results relate to future directions in the field

Enhanced perceptual processing of self-generated motion: Evidence from steady-state visual evoked potentials

The sense of agency emerges when our voluntary actions produce anticipated or predictable outcomes in the external world. It remains unclear how the sense of control also influences our perception of the external world. The present study examined perceptual processing of self-generated motion versus non-self-generated motion using steady-state visual evoked potentials (SSVEPs). Participants continuously moved their finger on a touchpad to trigger the movements of two shapes (Experiment 1) or two groups of dots (Experiment 2) on a monitor. Degree of control was manipulated by varying the spatial relation between finger movement and stimulus trajectory across conditions. However, the velocity, onset time, and offset time of visual stimuli always corresponded to participants' finger movement. Stimuli flickered at a frequency of either 7.5 Hz or 10 Hz, thus SSVEPs of these frequencies and their harmonics provided a frequency-tagged measurement of perceptual processing. Participants triggered the motion of all stimuli simultaneously, but had greater levels of control over some stimuli than over others. Their task was to detect a brief colour change on the border(s) of one shape (Experiment 1) or of one group of dots (Experiment 2). Although control over shapes/dots was irrelevant to the visual detection task, we found stronger SSVEPs for stimuli that were under a high level of control, compared with the stimuli that were under a low level of control. Our results suggest that the spatial regularity between self-generated movements and visual input boosted the neural responses underlying perceptual processing. Our results support the preactivation account of sensory attenuation, suggesting that perceptual processing of self-generated events is enhanced rather than inhibited

Selective Neuronal Entrainment to the Beat and Meter Embedded in a Musical Rhythm

Fundamental to the experience of music, beat and meter perception refers to the perception of periodicities while listening to music occurring within the frequency range of musical tempo. Here, we explored the spontaneous building of beat and meter hypothesized to emerge from the selective entrainment of neuronal populations at beat and meter frequencies. The electroencephalogram (EEG) was recorded while human participants listened to rhythms consisting of short sounds alternating with silences to induce a spontaneous perception of beat and meter. We found that the rhythmic stimuli elicited multiple steady state-evoked potentials (SS-EPs) observed in the EEG spectrum at frequencies corresponding to the rhythmic pattern envelope. Most importantly, the amplitude of the SS-EPs obtained at beat and meter frequencies were selectively enhanced even though the acoustic energy was not necessarily predominant at these frequencies. Furthermore, accelerating the tempo of the rhythmic stimuli so as to move away from the range of frequencies at which beats are usually perceived impaired the selective enhancement of SS-EPs at these frequencies. The observation that beat- and meter-related SS-EPs are selectively enhanced at frequencies compatible with beat and meter perception indicates that these responses do not merely reflect the physical structure of the sound envelope but, instead, reflect the spontaneous emergence of an internal representation of beat, possibly through a mechanism of selective neuronal entrainment within a resonance frequency range. Taken together, these results suggest that musical rhythms constitute a unique context to gain insight on general mechanisms of entrainment, from the neuronal level to individual level

Single-trial phase entrainment of theta oscillations in sensory regions predicts human associative memory performance

Episodic memories are rich in sensory information and often contain integrated information from different sensory modalities. For instance, we can store memories of a recent concert with visual and auditory impressions being integrated in one episode. Theta oscillations have recently been implicated in playing a causal role synchronizing and effectively binding the different modalities together in memory. However, an open question is whether momentary fluctuations in theta synchronization predict the likelihood of associative memory formation for multisensory events. To address this question we entrained the visual and auditory cortex at theta frequency (4 Hz) and in a synchronous or asynchronous manner by modulating the luminance and volume of movies and sounds at 4 Hz, with a phase offset at 0° or 180°. EEG activity from human subjects (both sexes) was recorded while they memorized the association between a movie and a sound. Associative memory performance was significantly enhanced in the 0° compared with the 180° condition. Source-level analysis demonstrated that the physical stimuli effectively entrained their respective cortical areas with a corresponding phase offset. The findings suggested a successful replication of a previous study (Clouter et al., 2017). Importantly, the strength of entrainment during encoding correlated with the efficacy of associative memory such that small phase differences between visual and auditory cortex predicted a high likelihood of correct retrieval in a later recall test. These findings suggest that theta oscillations serve a specific function in the episodic memory system: binding the contents of different modalities into coherent memory episodes

Exploring the neural entrainment to musical rhythms and meter : a steady-state evoked potential approach

Thèse de doctorat réalisé en cotutelle avec l'Université catholique de Louvain, Belgique (Faculté de médecine, Institut de Neuroscience)Percevoir et synchroniser ses mouvements à une pulsation régulière en musique est une capacité largement répandue chez l’Homme, et fondamentale aux comportements musicaux. La pulsation et la métrique en musique désignent généralement une organisation temporelle périodique perçue à partir de stimuli acoustiques complexes, et cette organisation perceptuelle implique souvent une mise en mouvement périodique spontanée du corps. Cependant, les mécanismes neuraux sous-tendant cette perception sont à l’heure actuelle encore méconnus. Le présent travail a donc eu pour objectif de développer une nouvelle approche expérimentale, inspirée par l’approche électrophysiologique des potentiels évoqués stationnaires, afin d’explorer les corrélats neuraux à la base de notre perception de la pulsation et de la métrique induite à l’écoute de rythmes musicaux. L’activité neurale évoquée en relation avec la perception d’une pulsation a été enregistrée par électroencéphalographie (EEG) chez des individus sains, dans divers contextes : (1) dans un contexte d’imagerie mentale d’une métrique appliquée de manière endogène sur un stimulus auditif, (2) dans un contexte d’induction spontanée d’une pulsation à l’écoute de patterns rythmiques musicaux, (3) dans un contexte d’interaction multisensorielle, et (4) dans un contexte de synchronisation sensorimotrice. Pris dans leur ensemble, les résultats de ces études corroborent l’hypothèse selon laquelle la perception de la pulsation en musique est sous-tendue par des processus de synchronisation et de résonance de l’activité neurale dans le cerveau humain. De plus, ces résultats suggèrent que l’approche développée dans le présent travail pourrait apporter un éclairage significatif pour comprendre les mécanismes neuraux de la perception de la pulsation et des rythmes musicaux, et, dans une perspective plus générale, pour explorer les mécanismes de synchronisation neurale.The ability to perceive a regular beat in music and synchronize to it is a widespread human skill. Fundamental to musical behavior, beat and meter refer to the perception of periodicities while listening to musical rhythms, and usually involve spontaneous entrainment to move on these periodicities. However, the neural mechanisms underlying entrainment to beat and meter in Humans remain unclear. The present work tests a novel experimental approach, inspired by the steady-state evoked potential method, to explore the neural dynamics supporting the perception of rhythmic inputs. Using human electroencephalography (EEG), neural responses to beat and meter were recorded in various contexts: (1) mental imagery of meter, (2) spontaneous induction of a beat from rhythmic patterns, (3) multisensory integration, and (4) sensorimotor synchronization. Our results support the view that entrainment and resonance phenomena subtend the processing of musical rhythms in the human brain. Furthermore, our results suggest that this novel approach could help investigating the link between the phenomenology of musical beat and meter and neurophysiological evidence of a bias towards periodicities arising under certain circumstances in the nervous system. Hence, entrainment to music provides an original framework to explore general entrainment phenomena occurring at various levels, from the inter-neural to the inter-individual level

Steady-state evoked potentials as an index of multisensory temporal binding.

Temporal congruency promotes perceptual binding of multisensory inputs. Here, we used EEG frequency-tagging to track cortical activities elicited by auditory and visual inputs separately, in the form of steady-state evoked potentials (SS-EPs). We tested whether SS-EPs could reveal a dynamic coupling of cortical activities related to the binding of auditory and visual inputs conveying synchronous vs. non-synchronous temporal periodicities, or beats. The temporally congruent audiovisual condition elicited markedly enhanced auditory and visual SS-EPs, as compared to the incongruent condition. Furthermore, an increased inter-trial phase coherence of both SS-EPs was observed in that condition. Taken together, these observations indicate that temporal congruency enhances the processing of multisensory inputs at sensory-specific stages of cortical processing, possibly through a dynamic binding by synchrony of the elicited activities and/or improved dynamic attending. Moreover, we show that EEG frequency-tagging with SS-EPs constitutes an effective tool to explore the neural dynamics of multisensory integration in the human brain

Processing resources and interplay among sensory modalities: an EEG investigation

The primary aim of the present thesis was to investigate how the human brain handles and distributes limited processing resources among different sensory modalities. Two main hypothesis have been conventionally proposed: (1) common processing resources shared among sensory modalities (supra-modal attentional system) or (2) independent processing resources for each sensory modality. By means of four EEG experiments, we tested whether putative competitive interactions between sensory modalities – regardless of attentional influences – are present in early sensory areas. We observed no competitive interactions between sensory modalities, supporting independent processing resources in early sensory areas. Consequently, we tested the influence of top-down attention on a cross-modal dual task. We found evidence for shared attentional resources between visual and tactile modalities. Taken together, our results point toward a hybrid model of inter-modal attention. Attentional processing resources seem to be controlled by a supra-modal attentional system, however, in early sensory areas, the absence of competitive interactions strongly reduces interferences between sensory modalities, thus providing a strong processing resource independence

From locomotion to dance and back : exploring rhythmic sensorimotor synchronization

Le rythme est un aspect important du mouvement et de la perception de l’environnement. Lorsque l’on danse, la pulsation musicale induit une activité neurale oscillatoire qui permet au système nerveux d’anticiper les évènements musicaux à venir. Le système moteur peut alors s’y synchroniser. Cette thèse développe de nouvelles techniques d’investigation des rythmes neuraux non strictement périodiques, tels que ceux qui régulent le tempo naturellement variable de la marche ou la perception rythmes musicaux. Elle étudie des réponses neurales reflétant la discordance entre ce que le système nerveux anticipe et ce qu’il perçoit, et qui sont nécessaire pour adapter la synchronisation de mouvements à un environnement variable. Elle montre aussi comment l’activité neurale évoquée par un rythme musical complexe est renforcée par les mouvements qui y sont synchronisés. Enfin, elle s’intéresse à ces rythmes neuraux chez des patients ayant des troubles de la marche ou de la conscience.Rhythms are central in human behaviours spanning from locomotion to music performance. In dance, self-sustaining and dynamically adapting neural oscillations entrain to the regular auditory inputs that is the musical beat. This entrainment leads to anticipation of forthcoming sensory events, which in turn allows synchronization of movements to the perceived environment. This dissertation develops novel technical approaches to investigate neural rhythms that are not strictly periodic, such as naturally tempo-varying locomotion movements and rhythms of music. It studies neural responses reflecting the discordance between what the nervous system anticipates and the actual timing of events, and that are critical for synchronizing movements to a changing environment. It also shows how the neural activity elicited by a musical rhythm is shaped by how we move. Finally, it investigates such neural rhythms in patient with gait or consciousness disorders