Electrophysiological signatures of conscious perception: The influence of cognitive, cortical and pathological states on multisensory integration

At any given moment, information reaches us via our different sensory systems. In order to navigate this multitude of information, associated information needs to be integrated to a coherent percept. In recent years, the hypothesis that synchronous neural oscillations play a prominent role in unisensory and multisensory processing has received substantial support. Current findings further convey the idea that local oscillations and functional connectivity reflect bottom-up as well as top-down processes during multisensory integration and perception. In the current work, I review recent findings on the role of neural oscillations for conscious multisensory perception. Subsequently, I present an integrative network model for multisensory integration that describes the cortical correlates of conscious multisensory perception, the influence of fluctuations of oscillatory neural activity on subsequent perception, and the influence of cognitive processes on neural oscillations and perception. I propose that neural oscillations in distinct, coexisting frequencies reflect the various processing steps underlying multisensory perception.Jederzeit erreichen uns Informationen über unsere verschiedenen Sinnesorgane und Wahrnehmungssysteme. Um in dieser Menge an Informationen den Überblick zu behalten, müssen zusammengehörige Informationen zu einer kohärente Wahrnehmung zusammengefügt werden. In den letzten Jahren hat die Hypothese, dass synchrone neuronale Oszillationen eine wichtige Rolle bei der Verarbeitung von unisensorischen und multisensorischen Reizen spielen, viel Unterstützung erfahren. Neueste Befunde befördern weiterhin die Idee, dass lokale Oszillationen und funktionale Konnektivität aufsteigende und absteigende Prozesse bei multisensorischer Integration und Wahrnehmung widerspiegeln. In dieser Arbeit werde ich einen Überblick über die neuesten Befunde zur Rolle neuronaler Oszillationen bei bewusster, multisensorischer Wahrnehmung geben. Anschließend werde ich ein integratives Netzwerkmodell multisensorischer Wahrnehmung präsentieren, welches die kortikalen Korrelate bewusster, multisensorischer Wahrnehmung, den Einfluss von Schwankungen oszillatorischer neuronaler Aktivität auf darauffolgende Wahrnehmung, sowie den Einfluss kognitiver Prozesse auf neuronale Oszillationen und Wahrnehmung beschreibt. Ich schlage vor, dass neuronale Oszillationen in umschriebenen, gleichzeitig aktiven Frequenzbändern die verschiedenen Verarbeitungsschritte widerspiegeln, welche multisensorischer Wahrnehmung zugrunde liegen

Causally Linking Neural Dominance to Perceptual Dominance in a Multisensory Conflict

When different senses are in conflict, one sense may dominate the perception of other sense, but it is not known whether the sensory cortex associated with the dominant modality exerts directional influence, at the functional brain level, over the sensory cortex associated with the dominated modality; in short, the link between sensory dominance and neuronal dominance is not established. In a task involving audio-visual conflict, using magnetoencephalography recordings in humans, we first demonstrated that the neuronal dominance – auditory cortex functionally influencing visual cortex – was associated with the sensory dominance – sound qualitatively altering visual perception. Further, we found that pre-stimulus auditory-to-visual connectivity could predict the perceptual outcome on a trial-by-trial basis. Subsequently, we performed an effective connectivity-guided neurofeedback electroencephalography experiment and showed that participants who were briefly trained to increase the neuronal dominance from auditory to visual cortex showed higher sensory, i.e. auditory, dominance during the conflict task immediately after the training. These results shed new light into the interactive neuronal nature of multisensory integration and open up exciting opportunities by enhancing or suppressing targeted mental functions subserved by effective connectivity

Temporal dynamics in the multisensory brain

Phd ThesisIn this work, I investigate the mechanisms with which the brain manages temporal coherence between sensory signals. An overview of relevant literature is given, and current theories about how sensory signals are combined in brain and behaviour are introduced. Key unknowns about the temporal dynamics of auditory-visual integration are identified and addressed within four investigations. In the first study, I assess whether cues to the onset of a auditory-visual pair affect sensitivity to their temporal asynchrony. It is shown that regularly timed cues shorten the temporal window of integration compared with irregular cues. This demonstrates that attention can affect how sensory signals are bound. In the second experiment, speech-like asynchronous stimuli are presented for an extended duration whilst perceptual simultaneity is monitored. In this manner, the time-course of temporal adaptation is tracked over time. Adaptation occurs when the presented asynchrony is visual-leading, but not when it is auditory-leading. This may suggest that temporal recalibration in the auditory-leading direction is not a consequence of adaptation. In the third investigation, the neural correlates of the time-course of temporal adaptation are measured. Increased activity in frontal and parietal areas occurred during perceptual asynchrony, this replicates previous work and further promotes that these regions provide top-down modulation of the mechanisms of temporal simultaneity. Increased activity is present in the posterior cingulate cortex whilst the brain is maintaining an adapted state, compared with during adaptation. This region may act as a con ict monitor and compensator for temporal asynchrony. Lastly, I investigate the extent to which a highly prevalent inhibitory neurotransmitter affects performance in a multisensory behavioural task. There is a possible correlation between the concentration of gamma-aminobutyric acid in the parietal lobe and the overall strength of integration effects. Finally, the impact and future directions of this work are discussed in the context of current literature.Wellcome Trust and the Institute of Neuroscienc

Double Flash Illusions: Current Findings and Future Directions

Twenty years ago, the first report on the sound-induced double flash illusion, a visual illusion induced by sound, was published. In this paradigm, participants are presented with different numbers of auditory and visual stimuli. In case of an incongruent number of auditory and visual stimuli, the influence of auditory information on visual perception can lead to the perception of the illusion. Thus, combining two auditory stimuli with one visual stimulus can induce the perception of two visual stimuli, the so-called fission illusion. Alternatively, combining one auditory stimulus with two visual stimuli can induce the perception of one visual stimulus, the so-called fusion illusion. Overall, current research shows that the illusion is a reliable indicator of multisensory integration. It has also been replicated using different stimulus combinations, such as visual and tactile stimuli. Importantly, the robustness of the illusion allows the widespread use for assessing multisensory integration across different groups of healthy participants and clinical populations and in various task setting. This review will give an overview of the experimental evidence supporting the illusion, the current state of research concerning the influence of cognitive processes on the illusion, the neural mechanisms underlying the illusion, and future research directions. Moreover, an exemplary experimental setup will be described with different options to examine perception, alongside code to test and replicate the illusion online or in the laboratory

Neural oscillatory signatures of auditory and audiovisual illusions

Questions of the relationship between human perception and brain activity can be approached from different perspectives: in the first, the brain is mainly regarded as a recipient and processor of sensory data. The corresponding research objective is to establish mappings of neural activity patterns and external stimuli. Alternatively, the brain can be regarded as a self-organized dynamical system, whose constantly changing state affects how incoming sensory signals are processed and perceived. The research reported in this thesis can chiefly be located in the second framework, and investigates the relationship between oscillatory brain activity and the perception of ambiguous stimuli. Oscillations are here considered as a mechanism for the formation of transient neural assemblies, which allows efficient information transfer. While the relevance of activity in distinct frequency bands for auditory and audiovisual perception is well established, different functional architectures of sensory integration can be derived from the literature. This dissertation therefore aims to further clarify the role of oscillatory activity in the integration of sensory signals towards unified perceptual objects, using illusion paradigms as tools of study. In study 1, we investigate the role of low frequency power modulations and phase alignment in auditory object formation. We provide evidence that auditory restoration is associated with a power reduction, while the registration of an additional object is reflected by an increase in phase locking. In study 2, we analyze oscillatory power as a predictor of auditory influence on visual perception in the sound-induced flash illusion. We find that increased beta-/ gamma-band power over occipitotemporal electrodes shortly before stimulus onset predicts the illusion, suggesting a facilitation of processing in polymodal circuits. In study 3, we address the question of whether visual influence on auditory perception in the ventriloquist illusion is reflected in primary sensory or higher-order areas. We establish an association between reduced theta-band power in mediofrontal areas and the occurrence of illusion, which indicates a top-down influence on sensory decision-making. These findings broaden our understanding of the functional relevance of neural oscillations by showing that different processing modes, which are reflected in specific spatiotemporal activity patterns, operate in different instances of sensory integration.Fragen nach dem Zusammenhang zwischen menschlicher Wahrnehmung und Hirnaktivität können aus verschiedenen Perspektiven adressiert werden: in der einen wird das Gehirn hauptsächlich als Empfänger und Verarbeiter von sensorischen Daten angesehen. Das entsprechende Forschungsziel wäre eine Zuordnung von neuronalen Aktivitätsmustern zu externen Reizen. Dieser Sichtweise gegenüber steht ein Ansatz, der das Gehirn als selbstorganisiertes dynamisches System begreift, dessen sich ständig verändernder Zustand die Verarbeitung und Wahrnehmung von sensorischen Signalen beeinflusst. Die Arbeiten, die in dieser Dissertation zusammengefasst sind, können vor allem in der zweitgenannten Forschungsrichtung verortet werden, und untersuchen den Zusammenhang zwischen oszillatorischer Hirnaktivität und der Wahrnehmung von mehrdeutigen Stimuli. Oszillationen werden hier als ein Mechanismus für die Formation von transienten neuronalen Zusammenschlüssen angesehen, der effizienten Informationstransfer ermöglicht. Obwohl die Relevanz von Aktivität in verschiedenen Frequenzbändern für auditorische und audiovisuelle Wahrnehmung gut belegt ist, können verschiedene funktionelle Architekturen der sensorischen Integration aus der Literatur abgeleitet werden. Das Ziel dieser Dissertation ist deshalb eine Präzisierung der Rolle oszillatorischer Aktivität bei der Integration von sensorischen Signalen zu einheitlichen Wahrnehmungsobjekten mittels der Nutzung von Illusionsparadigmen. In der ersten Studie untersuchen wir die Rolle von Leistung und Phasenanpassung in niedrigen Frequenzbändern bei der Formation von auditorischen Objekten. Wir zeigen, dass die Wiederherstellung von Tönen mit einer Reduktion der Leistung zusammenhängt, während die Registrierung eines zusätzlichen Objekts durch einen erhöhten Phasenangleich widergespiegelt wird. In der zweiten Studie analysieren wir oszillatorische Leistung als Prädiktor von auditorischem Einfluss auf visuelle Wahrnehmung in der sound-induced flash illusion. Wir stellen fest, dass erhöhte Beta-/Gamma-Band Leistung über occipitotemporalen Elektroden kurz vor der Reizdarbietung das Auftreten der Illusion vorhersagt, was auf eine Begünstigung der Verarbeitung in polymodalen Arealen hinweist. In der dritten Studie widmen wir uns der Frage, ob ein visueller Einfluss auf auditorische Wahrnehmung in der ventriloquist illusion sich in primären sensorischen oder übergeordneten Arealen widerspiegelt. Wir weisen einen Zusammenhang von reduzierter Theta-Band Leistung in mediofrontalen Arealen und dem Auftreten der Illusion nach, was einen top-down Einfluss auf sensorische Entscheidungsprozesse anzeigt. Diese Befunde erweitern unser Verständnis der funktionellen Bedeutung neuronaler Oszillationen, indem sie aufzeigen, dass verschiedene Verarbeitungsmodi, die sich in spezifischen räumlich-zeitlichen Aktivitätsmustern spiegeln, in verschiedenen Phänomenen von sensorischer Integration wirksam sind

The temporal sensitivity to the tactile-induced double flash illusion mediates the impact of beta oscillations on schizotypal personality traits

The coherent experience of the self and the world depends on the ability to integrate vs. segregate sensory information. Optimal temporal integration between the senses is mediated by oscillatory properties of neural activity. Previous research showed reduced temporal sensitivity to multisensory events in schizotypy, a personality trait linked to schizophrenia. Here we used the tactileinduced Double-Flash-Illusion (tDFI) to investigate the tactile-to-visual temporal sensitivity in schizotypy, as indexed by the temporal window of illusion (TWI) and its neural underpinnings. We measured EEG oscillations within the beta band, recently shown to correlate with the tDFI. We found individuals with higher schizotypal traits to have wider TWI and slower beta waves accounting for the temporal window within which they perceive the illusion. Our results indicate reduced tactile-to-visual temporal sensitivity to mediate the effect of slowed oscillatory beta activity on schizotypal personality traits. We conclude that slowed oscillatory patterns might constitute an early marker for psychosis proneness

Beta/Gamma Oscillations and Event-Related Potentials Indicate Aberrant Multisensory Processing in Schizophrenia

Recent behavioral and neuroimaging studies have suggested multisensory processing deficits in patients with schizophrenia (SCZ). Thus far, the neural mechanisms underlying these deficits are not well understood. Previous studies with unisensory stimulation have shown altered neural oscillations in SCZ. As such, altered oscillations could contribute to aberrant multisensory processing in this patient group. To test this assumption, we conducted an electroencephalography (EEG) study in 15 SCZ and 15 control participants in whom we examined neural oscillations and event-related potentials (ERPs) in the sound-induced flash illusion (SIFI). In the SIFI multiple auditory stimuli that are presented alongside a single visual stimulus can induce the illusory percept of multiple visual stimuli. In SCZ and control participants we compared ERPs and neural oscillations between trials that induced an illusion and trials that did not induce an illusion. On the behavioral level, SCZ (55.7%) and control participants (55.4%) did not significantly differ in illusion rates. The analysis of ERPs revealed diminished amplitudes and altered multisensory processing in SCZ compared to controls around 135 ms after stimulus onset. Moreover, the analysis of neural oscillations revealed altered 25–35 Hz power after 100 to 150 ms over occipital scalp for SCZ compared to controls. Our findings extend previous observations of aberrant neural oscillations in unisensory perception paradigms. They suggest that altered ERPs and altered occipital beta/gamma band power reflect aberrant multisensory processing in SCZ

Cognitive Impairment in Multiple Sclerosis Is Reflected by Increased Susceptibility to the Sound-Induced Flash Illusion

Objective: To determine whether the performance of multiple sclerosis (MS) patients in the sound-induced flash illusion (SiFi), a multisensory perceptual illusion, would reflect their cognitive impairment.Methods: We performed the SiFi task as well as an extensive neuropsychological testing in 95 subjects [39 patients with relapse-remitting MS (RRMS), 16 subjects with progressive multiple sclerosis (PMS) and 40 healthy control subjects (HC)].Results: MS patients reported more frequently the multisensory SiFi than HC. In contrast, there were no group differences in the control conditions. Essentially, patients with progressive type of MS continued to perceive the illusion at stimulus onset asynchronies (SOA) that were more than three times longer than the SOA at which the illusion was already disrupted for healthy controls. Furthermore, MS patients' degree of cognitive impairment measured with a broad neuropsychological battery encompassing tests for memory, attention, executive functions, and fluency was predicted by their performance in the SiFi task for the longest SOA of 500 ms.Conclusions: These findings support the notion that MS patients exhibit an altered multisensory perception in the SiFi task and that their susceptibility to the perceptual illusion is negatively correlated with their neuropsychological test performance. Since MS lesions affect white matter tracts and cortical regions which seem to be involved in the transfer and processing of both crossmodal and cognitive information, this might be one possible explanation for our findings. SiFi might be considered as a brief, non-expensive, language- and education-independent screening test for cognitive deficits in MS patients

Measuring neural excitation and inhibition in autism: different approaches, different findings and different interpretations.

The balance of neural excitation and inhibition (E/I balance) is often hypothesised to be altered in autism spectrum disorder (ASD). One widely held view is that excitation levels are elevated relative to inhibition in ASD. Understanding whether, and how, E/I balance may be altered in ASD is important given the recent interest in trialling pharmacological interventions for ASD which target inhibitory neurotransmitter function. Here we provide a critical review of evidence for E/I balance in ASD. We conclude that data from a number of domains provides support for alteration in excitation and inhibitory neurotransmission in ASD, but when considered collectively, the available literature provide little evidence to support claims for either a net increase in excitation or a net increase in inhibition. Strengths and limitations of available techniques are considered, and directions for future research discussed

A Neural “Tuning Curve” for Multisensory Experience and Cognitive-Perceptual Schizotypy

Our coherent perception of external events is enabled by the integration of inputs from different senses occurring within a range of temporal offsets known as the temporal binding window (TBW), which varies from person to person. A relatively wide TBW may increase the likelihood that stimuli originating from different environmental events are erroneously integrated and abnormally large TBW has been found in psychiatric disorders characterized by unusual perceptual experiences. Despite strong evidence of interindividual differences in TBW, both within clinical and nonclinical populations, the neurobiological underpinnings of this variability remain unclear. We adopted an integrated strategy linking TBW to temporal dynamics in functional magnetic resonance imaging (fMRI)-resting-state activity and cortical excitation/inhibition (E/I) balance, indexed by glutamate/Gamma-AminoButyric Acid (GABA) concentrations and common variation in glutamate and GABA genes in a healthy sample. Stronger resting-state longrange temporal correlations, indicated by larger power law exponent (PLE), in the auditory cortex, robustly predicted narrower audio-tactile TBW, which was in turn associated with lower cognitive-perceptual schizotypy. Furthermore, PLE was highest and TBW narrowest for individuals with intermediate levels of E/I balance, with shifts towards either extreme resulting in reduced multisensory temporal precision and increased schizotypy, effectively forming a neural ?tuning curve? for multisensory experience and schizophrenia risk. Our findings shed light on the neurobiological underpinnings of multisensory integration and its potentially clinically relevant inter-individual variability