Individual differences in alpha frequency drive crossmodal illusory perception

Perception routinely integrates inputs from different senses. Stimulus temporal proximity critically determines whether or not these inputs are bound together. Despite the temporal window of integration being a widely accepted notion, its neurophysiological substrate remains unclear. Many types of common audio-visual interactions occur within a time window of -100ms [1-5]. For example, in the sound- induced double-flash illusion, when two beeps are presented within -100ms together with one flash, a second illusory flash is often perceived [2]. Due to their intrinsic rhythmic nature, brain oscillations are one candidate mechanism for gating the temporal window of integration. Interestingly, occipital alpha-band oscillations cycle on average every -100ms with peak frequencies ranging between 8-14Hz (i.e. 120-60ms cycle). Moreover, presenting a brief tone can phase-reset such oscillations in visual cortex [6, 7]. Based on these observations, we hypothesized that the duration of each alpha cycle might provide the temporal unit to bind audio-visual events. Here we first recorded EEG while participants performed the sound-induced double-flash illusion task [4] and found positive correlation between individual alpha-frequency (IAF) peak and the size of the temporal window of the illusion. Participants then performed the same task while receiving occipital transcranial alternating current stimulation (tACS), to modulate oscillatory activity [8] either at their IAF or at off-peak alpha-frequencies (IAF±2Hz). Compared to IAF tACS, IAF-2Hz and IAF+2Hz tACS respectively enlarged and shrunk the temporal window of illusion, suggesting that alpha oscillations might represent the temporal unit of visual processing that cyclically gates perception and the neurophysiological substrate promoting audio-visual interactions

Sound-Induced Flash Illusion is Resistant to Feedback Training

A single flash accompanied by two auditory beeps tends to be perceived as two flashes (Shams et al. Nature 408:788, 2000, Cogn Brain Res 14:147–152, 2002). This phenomenon is known as ‘sound-induced flash illusion.’ Previous neuroimaging studies have shown that this illusion is correlated with modulation of activity in early visual cortical areas (Arden et al. Vision Res 43(23):2469–2478, 2003; Bhattacharya et al. NeuroReport 13:1727–1730, 2002; Shams et al. NeuroReport 12(17):3849–3852, 2001, Neurosci Lett 378(2):76–81, 2005; Watkins et al. Neuroimage 31:1247–1256, 2006, Neuroimage 37:572–578, 2007; Mishra et al. J Neurosci 27(15):4120–4131, 2007). We examined how robust the illusion is by testing whether the frequency of the illusion can be reduced by providing feedback. We found that the sound-induced flash illusion was resistant to feedback training, except when the amount of monetary reward was made dependent on accuracy in performance. However, even in the latter case the participants reported that they still perceived illusory two flashes even though they correctly reported single flash. Moreover, the feedback training effect seemed to disappear once the participants were no longer provided with feedback suggesting a short-lived refinement of discrimination between illusory and physical double flashes rather than vanishing of the illusory percept. These findings indicate that the effect of sound on the perceptual representation of visual stimuli is strong and robust to feedback training, and provide further evidence against decision factors accounting for the sound-induced flash illusion

Laminar fMRI: applications for cognitive neuroscience

The cortex is a massively recurrent network, characterized by feedforward and feedback connections between brain areas as well as lateral connections within an area. Feedforward, horizontal and feedback responses largely activate separate layers of a cortical unit, meaning they can be dissociated by lamina-resolved neurophysiological techniques. Such techniques are invasive and are therefore rarely used in humans. However, recent developments in high spatial resolution fMRI allow for non-invasive, in vivo measurements of brain responses specific to separate cortical layers. This provides an important opportunity to dissociate between feedforward and feedback brain responses, and investigate communication between brain areas at a more fine- grained level than previously possible in the human species. In this review, we highlight recent studies that successfully used laminar fMRI to isolate layer-specific feedback responses in human sensory cortex. In addition, we review several areas of cognitive neuroscience that stand to benefit from this new technological development, highlighting contemporary hypotheses that yield testable predictions for laminar fMRI. We hope to encourage researchers with the opportunity to embrace this development in fMRI research, as we expect that many future advancements in our current understanding of human brain function will be gained from measuring lamina-specific brain responses

Contextual modulation of primary visual cortex by auditory signals

Early visual cortex receives non-feedforward input from lateral and top-down connections (Muckli & Petro 2013 Curr. Opin. Neurobiol. 23, 195–201. (doi:10.1016/j.conb.2013.01.020)), including long-range projections from auditory areas. Early visual cortex can code for high-level auditory information, with neural patterns representing natural sound stimulation (Vetter et al. 2014 Curr. Biol. 24, 1256–1262. (doi:10.1016/j.cub.2014.04.020)). We discuss a number of questions arising from these findings. What is the adaptive function of bimodal representations in visual cortex? What type of information projects from auditory to visual cortex? What are the anatomical constraints of auditory information in V1, for example, periphery versus fovea, superficial versus deep cortical layers? Is there a putative neural mechanism we can infer from human neuroimaging data and recent theoretical accounts of cortex? We also present data showing we can read out high-level auditory information from the activation patterns of early visual cortex even when visual cortex receives simple visual stimulation, suggesting independent channels for visual and auditory signals in V1. We speculate which cellular mechanisms allow V1 to be contextually modulated by auditory input to facilitate perception, cognition and behaviour. Beyond cortical feedback that facilitates perception, we argue that there is also feedback serving counterfactual processing during imagery, dreaming and mind wandering, which is not relevant for immediate perception but for behaviour and cognition over a longer time frame. This article is part of the themed issue ‘Auditory and visual scene analysis’

Neural responses in parietal and occipital areas in response to visual events are modulated by prior multisensory stimuli

The effect of multi-modal vs uni-modal prior stimuli on the subsequent processing of a simple flash stimulus was studied in the context of the audio-visual 'flash-beep' illusion, in which the number of flashes a person sees is influenced by accompanying beep stimuli. EEG recordings were made while combinations of simple visual and audio-visual stimuli were presented. The experiments found that the electric field strength related to a flash stimulus was stronger when it was preceded by a multi-modal flash/beep stimulus, compared to when it was preceded by another uni-modal flash stimulus. This difference was found to be significant in two distinct timeframes--an early timeframe, from 130-160 ms, and a late timeframe, from 300-320 ms. Source localisation analysis found that the increased activity in the early interval was localised to an area centred on the inferior and superior parietal lobes, whereas the later increase was associated with stronger activity in an area centred on primary and secondary visual cortex, in the occipital lobe. The results suggest that processing of a visual stimulus can be affected by the presence of an immediately prior multisensory event. Relatively long-lasting interactions generated by the initial auditory and visual stimuli altered the processing of a subsequent visual stimulus.status: publishe

The Impact of Spatial Incongruence on an Auditory-Visual Illusion

The sound-induced flash illusion is an auditory-visual illusion--when a single flash is presented along with two or more beeps, observers report seeing two or more flashes. Previous research has shown that the illusion gradually disappears as the temporal delay between auditory and visual stimuli increases, suggesting that the illusion is consistent with existing temporal rules of neural activation in the superior colliculus to multisensory stimuli. However little is known about the effect of spatial incongruence, and whether the illusion follows the corresponding spatial rule. If the illusion occurs less strongly when auditory and visual stimuli are separated, then integrative processes supporting the illusion must be strongly dependant on spatial congruence. In this case, the illusion would be consistent with both the spatial and temporal rules describing response properties of multisensory neurons in the superior colliculus.status: publishe

Parietal disruption alters audiovisual binding in the sound-induced flash illusion

Selective attention and multisensory integration are fundamental to perception, but little is known about whether, or under what circumstances, these processes interact to shape conscious awareness. Here, we used transcranial magnetic stimulation (TMS) to investigate the causal role of attention-related brain networks in multisensory integration between visual and auditory stimuli in the sound-induced flash illusion. The flash illusion is a widely studied multisensory phenomenon in which a single flash of light is falsely perceived as multiple flashes in the presence of irrelevant sounds. We investigated the hypothesis that extrastriate regions involved in selective attention, specifically within the right parietal cortex, exert an influence on the multisensory integrative processes that cause the flash illusion. We found that disruption of the right angular gyrus, but not of the adjacent supramarginal gyrus or of a sensory control site, enhanced participants' veridical perception of the multisensory events, thereby reducing their susceptibility to the illusion. Our findings suggest that the same parietal networks that normally act to enhance perception of attended events also play a role in the binding of auditory and visual stimuli in the sound-induced flash illusion

Electrophysiological correlates and psychoacoustic characteristics of hearing-motion synaesthesia

People with hearing-motion synaesthesia experience sounds from moving or changing (e.g. flickering) visual stimuli. This phenomenon may be one of the most common forms of synaesthesia but it has rarely been studied and there are no studies of its neural basis. We screened for this in a sample of 200+ individuals, and estimated a prevalence of 4.2%. We also document its characteristics: it tends to be induced by physically moving stimuli (more so than static stimuli which imply motion or trigger illusory motion); and the psychoacoustic features are simple (e.g. “whooshing”) with some systematic correspondences to vision (e.g. faster movement is higher pitch). We demonstrate using event-related potentials that it emerges from early perceptual processing of vision. The synaesthetes have a higher amplitude motion-evoked N2 (165-185 msec), with some evidence of group differences as early as 55-75 msec. We discuss similarities between hearing-motion synaesthesia and previous observations that visual motion triggers auditory activity in the congenitally deaf. It is possible that both conditions reflect the maintenance of multisensory pathways found in early development that most people lose but can be retained in certain people in response to sensory deprivation (in the deaf) or, in people with normal hearing, as a result of other differences (e.g. genes predisposing to synaesthesia)

Neural oscillatory signatures of auditory and audiovisual illusions

Questions of the relationship between human perception and brain activity can be approached from different perspectives: in the first, the brain is mainly regarded as a recipient and processor of sensory data. The corresponding research objective is to establish mappings of neural activity patterns and external stimuli. Alternatively, the brain can be regarded as a self-organized dynamical system, whose constantly changing state affects how incoming sensory signals are processed and perceived. The research reported in this thesis can chiefly be located in the second framework, and investigates the relationship between oscillatory brain activity and the perception of ambiguous stimuli. Oscillations are here considered as a mechanism for the formation of transient neural assemblies, which allows efficient information transfer. While the relevance of activity in distinct frequency bands for auditory and audiovisual perception is well established, different functional architectures of sensory integration can be derived from the literature. This dissertation therefore aims to further clarify the role of oscillatory activity in the integration of sensory signals towards unified perceptual objects, using illusion paradigms as tools of study. In study 1, we investigate the role of low frequency power modulations and phase alignment in auditory object formation. We provide evidence that auditory restoration is associated with a power reduction, while the registration of an additional object is reflected by an increase in phase locking. In study 2, we analyze oscillatory power as a predictor of auditory influence on visual perception in the sound-induced flash illusion. We find that increased beta-/ gamma-band power over occipitotemporal electrodes shortly before stimulus onset predicts the illusion, suggesting a facilitation of processing in polymodal circuits. In study 3, we address the question of whether visual influence on auditory perception in the ventriloquist illusion is reflected in primary sensory or higher-order areas. We establish an association between reduced theta-band power in mediofrontal areas and the occurrence of illusion, which indicates a top-down influence on sensory decision-making. These findings broaden our understanding of the functional relevance of neural oscillations by showing that different processing modes, which are reflected in specific spatiotemporal activity patterns, operate in different instances of sensory integration.Fragen nach dem Zusammenhang zwischen menschlicher Wahrnehmung und Hirnaktivität können aus verschiedenen Perspektiven adressiert werden: in der einen wird das Gehirn hauptsächlich als Empfänger und Verarbeiter von sensorischen Daten angesehen. Das entsprechende Forschungsziel wäre eine Zuordnung von neuronalen Aktivitätsmustern zu externen Reizen. Dieser Sichtweise gegenüber steht ein Ansatz, der das Gehirn als selbstorganisiertes dynamisches System begreift, dessen sich ständig verändernder Zustand die Verarbeitung und Wahrnehmung von sensorischen Signalen beeinflusst. Die Arbeiten, die in dieser Dissertation zusammengefasst sind, können vor allem in der zweitgenannten Forschungsrichtung verortet werden, und untersuchen den Zusammenhang zwischen oszillatorischer Hirnaktivität und der Wahrnehmung von mehrdeutigen Stimuli. Oszillationen werden hier als ein Mechanismus für die Formation von transienten neuronalen Zusammenschlüssen angesehen, der effizienten Informationstransfer ermöglicht. Obwohl die Relevanz von Aktivität in verschiedenen Frequenzbändern für auditorische und audiovisuelle Wahrnehmung gut belegt ist, können verschiedene funktionelle Architekturen der sensorischen Integration aus der Literatur abgeleitet werden. Das Ziel dieser Dissertation ist deshalb eine Präzisierung der Rolle oszillatorischer Aktivität bei der Integration von sensorischen Signalen zu einheitlichen Wahrnehmungsobjekten mittels der Nutzung von Illusionsparadigmen. In der ersten Studie untersuchen wir die Rolle von Leistung und Phasenanpassung in niedrigen Frequenzbändern bei der Formation von auditorischen Objekten. Wir zeigen, dass die Wiederherstellung von Tönen mit einer Reduktion der Leistung zusammenhängt, während die Registrierung eines zusätzlichen Objekts durch einen erhöhten Phasenangleich widergespiegelt wird. In der zweiten Studie analysieren wir oszillatorische Leistung als Prädiktor von auditorischem Einfluss auf visuelle Wahrnehmung in der sound-induced flash illusion. Wir stellen fest, dass erhöhte Beta-/Gamma-Band Leistung über occipitotemporalen Elektroden kurz vor der Reizdarbietung das Auftreten der Illusion vorhersagt, was auf eine Begünstigung der Verarbeitung in polymodalen Arealen hinweist. In der dritten Studie widmen wir uns der Frage, ob ein visueller Einfluss auf auditorische Wahrnehmung in der ventriloquist illusion sich in primären sensorischen oder übergeordneten Arealen widerspiegelt. Wir weisen einen Zusammenhang von reduzierter Theta-Band Leistung in mediofrontalen Arealen und dem Auftreten der Illusion nach, was einen top-down Einfluss auf sensorische Entscheidungsprozesse anzeigt. Diese Befunde erweitern unser Verständnis der funktionellen Bedeutung neuronaler Oszillationen, indem sie aufzeigen, dass verschiedene Verarbeitungsmodi, die sich in spezifischen räumlich-zeitlichen Aktivitätsmustern spiegeln, in verschiedenen Phänomenen von sensorischer Integration wirksam sind