Neural oscillatory signatures of auditory and audiovisual illusions

Questions of the relationship between human perception and brain activity can be approached from different perspectives: in the first, the brain is mainly regarded as a recipient and processor of sensory data. The corresponding research objective is to establish mappings of neural activity patterns and external stimuli. Alternatively, the brain can be regarded as a self-organized dynamical system, whose constantly changing state affects how incoming sensory signals are processed and perceived. The research reported in this thesis can chiefly be located in the second framework, and investigates the relationship between oscillatory brain activity and the perception of ambiguous stimuli. Oscillations are here considered as a mechanism for the formation of transient neural assemblies, which allows efficient information transfer. While the relevance of activity in distinct frequency bands for auditory and audiovisual perception is well established, different functional architectures of sensory integration can be derived from the literature. This dissertation therefore aims to further clarify the role of oscillatory activity in the integration of sensory signals towards unified perceptual objects, using illusion paradigms as tools of study. In study 1, we investigate the role of low frequency power modulations and phase alignment in auditory object formation. We provide evidence that auditory restoration is associated with a power reduction, while the registration of an additional object is reflected by an increase in phase locking. In study 2, we analyze oscillatory power as a predictor of auditory influence on visual perception in the sound-induced flash illusion. We find that increased beta-/ gamma-band power over occipitotemporal electrodes shortly before stimulus onset predicts the illusion, suggesting a facilitation of processing in polymodal circuits. In study 3, we address the question of whether visual influence on auditory perception in the ventriloquist illusion is reflected in primary sensory or higher-order areas. We establish an association between reduced theta-band power in mediofrontal areas and the occurrence of illusion, which indicates a top-down influence on sensory decision-making. These findings broaden our understanding of the functional relevance of neural oscillations by showing that different processing modes, which are reflected in specific spatiotemporal activity patterns, operate in different instances of sensory integration.Fragen nach dem Zusammenhang zwischen menschlicher Wahrnehmung und Hirnaktivität können aus verschiedenen Perspektiven adressiert werden: in der einen wird das Gehirn hauptsächlich als Empfänger und Verarbeiter von sensorischen Daten angesehen. Das entsprechende Forschungsziel wäre eine Zuordnung von neuronalen Aktivitätsmustern zu externen Reizen. Dieser Sichtweise gegenüber steht ein Ansatz, der das Gehirn als selbstorganisiertes dynamisches System begreift, dessen sich ständig verändernder Zustand die Verarbeitung und Wahrnehmung von sensorischen Signalen beeinflusst. Die Arbeiten, die in dieser Dissertation zusammengefasst sind, können vor allem in der zweitgenannten Forschungsrichtung verortet werden, und untersuchen den Zusammenhang zwischen oszillatorischer Hirnaktivität und der Wahrnehmung von mehrdeutigen Stimuli. Oszillationen werden hier als ein Mechanismus für die Formation von transienten neuronalen Zusammenschlüssen angesehen, der effizienten Informationstransfer ermöglicht. Obwohl die Relevanz von Aktivität in verschiedenen Frequenzbändern für auditorische und audiovisuelle Wahrnehmung gut belegt ist, können verschiedene funktionelle Architekturen der sensorischen Integration aus der Literatur abgeleitet werden. Das Ziel dieser Dissertation ist deshalb eine Präzisierung der Rolle oszillatorischer Aktivität bei der Integration von sensorischen Signalen zu einheitlichen Wahrnehmungsobjekten mittels der Nutzung von Illusionsparadigmen. In der ersten Studie untersuchen wir die Rolle von Leistung und Phasenanpassung in niedrigen Frequenzbändern bei der Formation von auditorischen Objekten. Wir zeigen, dass die Wiederherstellung von Tönen mit einer Reduktion der Leistung zusammenhängt, während die Registrierung eines zusätzlichen Objekts durch einen erhöhten Phasenangleich widergespiegelt wird. In der zweiten Studie analysieren wir oszillatorische Leistung als Prädiktor von auditorischem Einfluss auf visuelle Wahrnehmung in der sound-induced flash illusion. Wir stellen fest, dass erhöhte Beta-/Gamma-Band Leistung über occipitotemporalen Elektroden kurz vor der Reizdarbietung das Auftreten der Illusion vorhersagt, was auf eine Begünstigung der Verarbeitung in polymodalen Arealen hinweist. In der dritten Studie widmen wir uns der Frage, ob ein visueller Einfluss auf auditorische Wahrnehmung in der ventriloquist illusion sich in primären sensorischen oder übergeordneten Arealen widerspiegelt. Wir weisen einen Zusammenhang von reduzierter Theta-Band Leistung in mediofrontalen Arealen und dem Auftreten der Illusion nach, was einen top-down Einfluss auf sensorische Entscheidungsprozesse anzeigt. Diese Befunde erweitern unser Verständnis der funktionellen Bedeutung neuronaler Oszillationen, indem sie aufzeigen, dass verschiedene Verarbeitungsmodi, die sich in spezifischen räumlich-zeitlichen Aktivitätsmustern spiegeln, in verschiedenen Phänomenen von sensorischer Integration wirksam sind

Similar EEG Activity Patterns During Experimentally-Induced Auditory Illusions and Veridical Perceptions

Hallucinations and illusions are two instances of perceptual experiences illustrating how perception might diverge from external sensory stimulations and be generated or altered based on internal brain states. The occurrence of these phenomena is not constrained to patient populations. Similar experiences can be elicited in healthy subjects by means of suitable experimental procedures. Studying the neural mechanisms underlying these experiences not only has the potential to expand our understanding of the brain's perceptual machinery but also of how it might get impaired. In the current study, we employed an auditory signal detection task to induce auditory illusions by presenting speech snippets at near detection threshold intensity embedded in noise. We investigated the neural correlates of auditory false perceptions by examining the EEG activity preceding the responses in speech absent (false alarm, FA) trials and comparing them to speech present (hit) trials. The results of the comparison of event-related potentials (ERPs) in the activation period vs. baseline revealed the presence of an early negativity (EN) and a late positivity (LP) similar in both hits and FAs, which were absent in misses, correct rejections (CR) and control button presses (BPs). We postulate that the EN and the LP might represent the auditory awareness negativity (AAN) and centro-parietal positivity (CPP) or P300, respectively. The event-related spectral perturbations (ERSPs) exhibited a common power enhancement in low frequencies

Sounds in noise: Behavioral and neural studies of illusory continuity and discontinuity

ability to parse an auditory scene into meaningful components varies greatly between individuals; some are able to parse out and write down competing musical pieces while others struggle to understand each word whenever they have to converse in a noisy environment. Using a simple discrimination task, healthy, normally-heari ng adult participants were asked to judge whether a pure tone (with or without amplitude modulation) was continuous or contained a gap. One quarter of the participants consistently heard a gap when none was present, if the tone was accompanied by a higher-frequency noise burst with a lower edge beginning one octave away from the tone (that did not have any energy overlapping the tone). This novel form of informational masking (perceptual interference between components with non-overlapping sound energy) was named 'illusory auditory discontinuity\u2019. The phenomenon appears to reflect natural differences in auditory processing rather than differences in decision-making strategies because: (1) susceptibility to illusory discontinuity correlates with individual differences in auditory streaming (measured using a classical ABA sequential paradigm); and (2) electroencephalographic responses elicited by tones overlaid by short noise bursts (when these sounds are not the focus of attention) are significantly correlated with the occurrence of illusory auditory discontinuity in both an early event-related potential (ERP) component (40-66 ms), and a later ERP component (270-350 ms) after noise onset. Participants prone to illusory discontinuity also tended not to perceive the \u2018auditory continuity illusion\u2019 (in which a tone is heard continuing under a burst of noise centered on the tone frequency that completely masks it) at short noise durations, but reliably perceived the auditory continuity illusion at longer noise durations. These results suggest that a number of attributes describing how individuals differentially parse complex auditory scenes are related to individual differences in two potentially independent attributes of neural processing, reflected here by EEG waveform differences at ~50 msec and ~300 msec after noise onset. Neural correlates of the auditory continuity illusion were also investigated by adjusting masker loudness, so that when listeners were given physically identical stimuli, they correctly detected the gap in a target tone on some trials, while on other trials they reported the tone as continuous (experiencing illusory continuity). High er power of low-frequency EEG activity (in the delta-theta range, <6 Hz) was observed prior to the onset of tones that were subsequently judged as discontinuous, with no other consistent EEG differences found after the onset of tones. These data suggest that the occurrence of the continuity illusion may depend on the brain state that exists immediately before a trial begins

Defining Auditory-Visual Objects: Behavioral Tests and Physiological Mechanisms

Crossmodal integration is a term applicable to many phenomena in which one sensory modality influences task performance or perception in another sensory modality. We distinguish the term binding as one that should be reserved specifically for the process that underpins perceptual object formation. To unambiguously differentiate binding form other types of integration, behavioral and neural studies must investigate perception of a feature orthogonal to the features that link the auditory and visual stimuli. We argue that supporting true perceptual binding (as opposed to other processes such as decision-making) is one role for cross-sensory influences in early sensory cortex. These early multisensory interactions may therefore form a physiological substrate for the bottom-up grouping of auditory and visual stimuli into auditory-visual (AV) objects

Stimulus Familiarity Affects Perceptual Restoration in the European Starling (Sturnus vulgaris)

Background: Humans can easily restore a speech signal that is temporally masked by an interfering sound (e.g., a cough masking parts of a word in a conversation), and listeners have the illusion that the speech continues through the interfering sound. This perceptual restoration for human speech is affected by prior experience. Here we provide evidence for perceptual restoration in complex vocalizations of a songbird that are acquired by vocal learning in a similar way as humans learn their language. Methodology/Principal Findings: European starlings were trained in a same/different paradigm to report salient differences between successive sounds. The birds ’ response latency for discriminating between a stimulus pair is an indicator for the salience of the difference, and these latencies can be used to evaluate perceptual distances using multidimensional scaling. For familiar motifs the birds showed a large perceptual distance if discriminating between song motifs that were muted for brief time periods and complete motifs. If the muted periods were filled with noise, the perceptual distance was reduced. For unfamiliar motifs no such difference was observed. Conclusions/Significance: The results suggest that starlings are able to perceptually restore partly masked sounds and, similarly to humans, rely on prior experience. They may be a suitable model to study the mechanism underlying experiencedependent perceptual restoration

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Mind-life continuity: a qualitative study of conscious experience

There are two fundamental models to understanding the phenomenon of natural life. One is thecomputational model, which is based on the symbolic thinking paradigm. The other is the biologicalorganism model. The common difficulty attributed to these paradigms is that their reductive tools allowthe phenomenological aspects of experience to remain hidden behind yes/no responses (behavioraltests), or brain ‘pictures’ (neuroimaging). Hence, one of the problems regards how to overcome meth-odological difficulties towards a non-reductive investigation of conscious experience. It is our aim in thispaper to show how cooperation between Eastern and Western traditions may shed light for a non-reductive study of mind and life. This study focuses on the first-person experience associated withcognitive and mental events. We studied phenomenal data as a crucial fact for the domain of livingbeings, which, we expect, can provide the ground for a subsequent third-person study. The interventionwith Jhana meditation, and its qualitative assessment, provided us with experiential profiles based uponsubjects' evaluations of their own conscious experiences. The overall results should move towards anintegrated or global perspective on mind where neither experience nor external mechanisms have thefinal wor