Hearing through your eyes: neural basis of audiovisual cross-activation, revealed by transcranial alternating current stimulation

Some people experience auditory sensations when seeing visual flashes or movements. This prevalent synaesthesia-like ‘visual-evoked auditory response’ (vEAR) could result either from over-exuberant cross-activation between brain areas, and/or reduced inhibition of normally-occurring cross-activation. We have used transcranial alternating current stimulation (tACS) to test these theories. We applied tACS at 10Hz (alpha-band frequency) or 40Hz (gamma-band), bilaterally either to temporal or occipital sites, while measuring same/different discrimination of paired auditory (A) versus visual (V) 'Morse code' sequences. At debriefing, participants were classified as vEAR or non-vEAR depending on whether they reported 'hearing' the silent flashes. In non-vEAR participants, temporal 10Hz tACS caused impairment of A performance, which correlated with improved V; conversely under occipital tACS, poorer V performance correlated with improved A. This reciprocal pattern suggests that sensory cortices are normally mutually inhibitory, and that alpha-frequency tACS may bias the balance of competition between them. vEAR participants showed no tACS effects, consistent with reduced inhibition, or enhanced cooperation between modalities. In addition, temporal 40Hz tACS impaired V performance, specifically in individuals who showed a performance advantage for V (relative to A). Gamma-frequency tACS may therefore modulate the ability of these individuals to benefit from recoding flashes into the auditory modality, possibly by disrupting cross-activation of auditory areas by visual stimulation. Our results support both theories, suggesting that vEAR may depend on disinhibition of normally-occurring sensory cross-activation, which may be expressed more strongly in some individuals. Furthermore, endogenous alpha and gamma-frequency oscillations may function respectively to inhibit or promote this cross-activation

The sound of concepts: The link between auditory and conceptual brain systems

Concepts in long-term memory are important building blocks of human cognition and are the basis for object recognition, language and thought. While it is well accepted that concepts are comprised of features related to sensory object attributes, it is still unclear how these features are represented in the brain. Of central interest is whether concepts are essentially grounded in perception. This would imply a common neuroanatomical substrate for perceptual and conceptual processing. Here we show using functional magnetic resonance imaging and recordings of event-related potentials that acoustic conceptual features rapidly recruit auditory areas even when implicitly presented through visual words. Recognizing words denoting objects for which acoustic features are highly relevant (e.g. "telephone") suffices to ignite cell assemblies in the posterior superior and middle temporal gyrus (pSTG/MTG) that were also activated by listening to real sounds. Activity in pSTG/MTG had an onset of 150 ms and increased parametrically as a function of acoustic feature relevance. Both findings suggest a conceptual origin of this effect rather than post-conceptual strategies such as imagery. The presently demonstrated link between auditory and conceptual brain systems parallels observations in other memory systems suggesting that modality-specificity represents a general organizational principle in cortical memory representation. The understanding of concepts as a partial reinstatement of brain activity during perception stresses the necessity of rich sensory experiences for concept acquisition. The modality-specific nature of concepts could also explain the difficulties in achieving a consensus about overall definitions of abstract concepts such as freedom or justice unless embedded in a concrete, experienced situation

Laminar fMRI: applications for cognitive neuroscience

The cortex is a massively recurrent network, characterized by feedforward and feedback connections between brain areas as well as lateral connections within an area. Feedforward, horizontal and feedback responses largely activate separate layers of a cortical unit, meaning they can be dissociated by lamina-resolved neurophysiological techniques. Such techniques are invasive and are therefore rarely used in humans. However, recent developments in high spatial resolution fMRI allow for non-invasive, in vivo measurements of brain responses specific to separate cortical layers. This provides an important opportunity to dissociate between feedforward and feedback brain responses, and investigate communication between brain areas at a more fine- grained level than previously possible in the human species. In this review, we highlight recent studies that successfully used laminar fMRI to isolate layer-specific feedback responses in human sensory cortex. In addition, we review several areas of cognitive neuroscience that stand to benefit from this new technological development, highlighting contemporary hypotheses that yield testable predictions for laminar fMRI. We hope to encourage researchers with the opportunity to embrace this development in fMRI research, as we expect that many future advancements in our current understanding of human brain function will be gained from measuring lamina-specific brain responses

An EMG study of the lip muscles during covert auditory verbal hallucinations in schizophrenia

Purpose: Auditory verbal hallucinations (AVHs) are speech perceptions in the absence of a external stimulation. An influential theoretical account of AVHs in schizophrenia claims that a deficit in inner speech monitoring would cause the verbal thoughts of the patient to be perceived as external voices. The account is based on a predictive control model, in which verbal self-monitoring is implemented. The aim of this study was to examine lip muscle activity during AVHs in schizophrenia patients, in order to check whether inner speech occurred. Methods: Lip muscle activity was recorded during covert AVHs (without articulation) and rest. Surface electromyography (EMG) was used on eleven schizophrenia patients. Results: Our results show an increase in EMG activity in the orbicularis oris inferior muscle, during covert AVHs relative to rest. This increase is not due to general muscular tension since there was no increase of muscular activity in the forearm muscle. Conclusion: This evidence that AVHs might be self-generated inner speech is discussed in the framework of a predictive control model. Further work is needed to better describe how the inner speech monitoring dysfunction occurs and how inner speech is controlled and monitored. This will help better understanding how AVHs occur

Cortico-muscular coherence in sensorimotor synchronisation

This thesis sets out to investigate the neuro-muscular control mechanisms underlying the ubiquitous phenomenon of sensorimotor synchronisation (SMS). SMS is the coordination of movement to external rhythms, and is commonly observed in everyday life. A large body of research addresses the processes underlying SMS at the levels of behaviour and brain. Comparatively, little is known about the coupling between neural and behavioural processes, i.e. neuro-muscular processes. Here, the neuro-muscular processes underlying SMS were investigated in the form of cortico-muscular coherence measured based on Electroencephalography (EEG) and Electromyography (EMG) recorded in human healthy participants. These neuro-muscular processes were investigated at three levels of engagement: passive listening and observation of rhythms in the environment, imagined SMS, and executed SMS, which resulted in the testing of three hypotheses: (i) Rhythms in the environment, such as music, spontaneously modulate cortico-muscular coupling, (ii) Movement intention modulates cortico-muscular coupling, and (iii) Cortico-muscular coupling is dynamically modulated during SMS time-locked to the stimulus rhythm. These three hypotheses were tested through two studies that used Electroencephalography (EEG) and Electromyography (EMG) recordings to measure Cortico-muscular coherence (CMC). First, CMC was tested during passive music listening, to test whether temporal and spectral properties of music stimuli known to induce groove, i.e., the subjective experience of wanting to move, can spontaneously modulate the overall strength of the communication between the brain and the muscles. Second, imagined and executed movement synchronisation was used to investigate the role of movement intention and dynamics on CMC. The two studies indicate that both top-down, and somatosensory and/or proprioceptive processes modulate CMC during SMS tasks. Although CMC dynamics might be linked to movement dynamics, no direct correlation between movement performance and CMC was found. Furthermore, purely passive auditory or visual rhythmic stimulation did not affect CMC. Together, these findings thus indicate that movement intention and active engagement with rhythms in the environment might be critical in modulating CMC. Further investigations of the mechanisms and function of CMC are necessary, as they could have important implications for clinical and elderly populations, as well as athletes, where optimisation of motor control is necessary to compensate for impaired movement or to achieve elite performance

Modulation of Visual Cortical Excitability by Working Memory: Effect of Luminance Contrast of Mental Imagery

Although much is known about the impact of stimulus properties such as luminance contrast, spatial frequency, and orientation on visually evoked neural activity, much less is known about how they modulate neural activity when they are properties of a mental image held in working memory (WM). Here we addressed this question by investigating how a parametric manipulation of an imagined stimulus attribute affects neuronal excitability in the early visual cortex. We manipulated luminance contrast, a stimulus property known to strongly affect the magnitude of neuronal responses in early visual areas. Luminance contrast modulated neuronal excitability, as assessed by the frequency of phosphenes induced by transcranial magnetic stimulation (TMS) with the exact nature of this modulation depending on TMS intensity. These results point to a strong overlap in the neuronal processes underlying visual perception and mental imagery: not only does WM maintenance selectively engage neurons which are tuned to the maintained attribute (as has previously been shown), but the extent to which those neurons are activated depends on the image contrast (as is the case with visually evoked responses). From a methodological viewpoint, these results suggest that assessment of visual cortical excitability using TMS is affected by the TMS intensity used to probe the neuronal population

Assessing the impact of emotion in dual pathway models of sensory processing.

In our daily environment, we are constantly encountering an endless stream of information which we must be able to sort and prioritize. Some of the features that influence this are the emotional nature of stimuli and the emotional context of events. Emotional information is often given preferential access to neurocognitive resources, including within sensory processing systems. Interestingly, both auditory and visual systems are divided into dual processing streams; a ventral object identity/perception stream and a dorsal object location/action stream. While effects of emotion on the ventral streams are relatively well defined, its effect on dorsal stream processes remains unclear. The present thesis aimed to investigate the impact of emotion on sensory systems within a dual pathway framework of sensory processing. Study I investigated the role of emotion during auditory localization. While undergoing fMRI, participants indicated the location of an emotional or non-emotional sound within an auditory virtual environment. This revealed that the neurocognitive structures displaying activation modulated by emotion were not the same as those modulated by sound location. Emotion was represented in regions associated with the putative auditory ‘what’ but not ‘where’ stream. Study II examined the impact of emotion on ostensibly similar localization behaviours mediated differentially by the dorsal versus ventral visual processing stream. Ventrally-mediated behaviours were demonstrated to be impacted by the emotional context of a trial, while dorsally-mediated behaviours were not. For Study III, a motion-aftereffect paradigm was used to investigate the impact of emotion on visual area V5/MT+. This area, traditionally believed to be involved in dorsal stream processing, has a number of characteristics similar to a ventral stream structure. It was discovered that V5/MT+ activity was modulated both by presence of perceptual motion and emotional content of an image. In addition, this region displayed patterns of functional connectivity with the amygdala that were significantly modulated by emotion. Together, these results suggest that emotional information modulates neural processing within ventral sensory processing streams, but not dorsal processing streams. These findings are discussed with respect to current models of emotional and sensory processing, including amygdala connections to sensory cortices and emotional effects on cognition and behaviour

Orientation in space using the sense of smell

Several studies reported that respiration interacts with olfactory perception. Therefore, in the pilot study of this experiment series human breathing was investigated during an olfactory experiment. Breathing parameters (respiratory minute volume, respiratory amplitude, and breathing rate) were quantified in response to odor stimulation and olfactory imagery. We provide evidence that respiration changed during smelling and during olfactory imagery in comparison to the baseline condition. In conclusion, olfactory perception and olfactory imagery both have an impact on the human respiratory profile, which is hypothesized to be based on a common underlying mechanism named sniffing. Our findings underline that for certain aspects of olfactory research it may be necessary to control and/or monitor respiration during olfactory stimulation. The human ability to localize odors has been investigated in a limited number of studies, but the findings are contradictory. We hypothesized that this was mainly due to differential effects of olfactory and trigeminal stimulation. Only few substances excite selectively the olfactory system. One of them is hydrogen sulphide (H2S). In contrast, most odorants stimulate both olfactory and trigeminal receptors of the nasal mucosa. The main goal of this study was to test the human ability to localize substances, which excite the olfactory system selectively. For this purpose we performed localization experiment using low and high concentrations of the pure odorant H2S, the olfactory-trigeminal substance isoamyl acetate (IAA), and the trigeminal substance carbon dioxide (CO2). In preparation for the localization study a detection experiment was carried out to ensure that subjects perceived the applied stimuli consciously. The aim of the detection study was to quantify the human sensitivity in response to stimulation with H2S, IAA, and CO2. We tested healthy subjects using an event-related experimental design. The olfactory stimulation was performed using an olfactometer. The results showed that humans are able to detect H2S in low concentration (2 ppm) with moderate sensitivity, and possess a high sensitivity in response to stimulation with 8ppm H2S, 50% v/v CO2, and 17.5% v/v IAA. The localization experiment revealed that subjects can localize H2S neither in low nor in high concentrations. In contrast to that, subjects possess an ability to localize both IAA and CO2 stimuli. These results clearly demonstrate that humans are able to localize odorants which excite the trigeminal system, but they are not able to localize odors that stimulate the olfactory system exclusively, in spite of consciously perceiving the stimuli

Sensorimotor Modulations by Cognitive Processes During Accurate Speech Discrimination: An EEG Investigation of Dorsal Stream Processing

Internal models mediate the transmission of information between anterior and posterior regions of the dorsal stream in support of speech perception, though it remains unclear how this mechanism responds to cognitive processes in service of task demands. The purpose of the current study was to identify the influences of attention and working memory on sensorimotor activity across the dorsal stream during speech discrimination, with set size and signal clarity employed to modulate stimulus predictability and the time course of increased task demands, respectively. Independent Component Analysis of 64–channel EEG data identified bilateral sensorimotor mu and auditory alpha components from a cohort of 42 participants, indexing activity from anterior (mu) and posterior (auditory) aspects of the dorsal stream. Time frequency (ERSP) analysis evaluated task-related changes in focal activation patterns with phase coherence measures employed to track patterns of information flow across the dorsal stream. ERSP decomposition of mu clusters revealed event-related desynchronization (ERD) in beta and alpha bands, which were interpreted as evidence of forward (beta) and inverse (alpha) internal modeling across the time course of perception events. Stronger pre-stimulus mu alpha ERD in small set discrimination tasks was interpreted as more efficient attentional allocation due to the reduced sensory search space enabled by predictable stimuli. Mu-alpha and mu-beta ERD in peri- and post-stimulus periods were interpreted within the framework of Analysis by Synthesis as evidence of working memory activity for stimulus processing and maintenance, with weaker activity in degraded conditions suggesting that covert rehearsal mechanisms are sensitive to the quality of the stimulus being retained in working memory. Similar ERSP patterns across conditions despite the differences in stimulus predictability and clarity, suggest that subjects may have adapted to tasks. In light of this, future studies of sensorimotor processing should consider the ecological validity of the tasks employed, as well as the larger cognitive environment in which tasks are performed. The absence of interpretable patterns of mu-auditory coherence modulation across the time course of speech discrimination highlights the need for more sensitive analyses to probe dorsal stream connectivity