The perception of dynamic and static facial expressions of happiness and disgust investigated by ERPs and fMRI constrained source analysis

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach

Temporal dynamics of amygdala response to emotion- and action-relevance

It has been proposed that the human amygdala may not only encode the emotional value of sensory events, but more generally mediate the appraisal of their relevance for the individual's goals, including relevance for action or task-based needs. However, emotional and non-emotional/action-relevance might drive amygdala activity through distinct neural signals, and the relative timing of both kinds of responses remains undetermined. Here, we recorded intracranial event-related potentials (iERPs) from nine amygdalae of patients undergoing epilepsy surgery, while they performed variants of a Go/NoGo task with faces and abstract shapes, where emotion- and action-relevance were orthogonally manipulated. Our results revealed early amygdala responses to emotion facial expressions starting ~130ms after stimulus-onset. Importantly, the amygdala responded to action-relevance not only with face stimuli but also with abstract shapes (squares), and these relevance effects consistently occurred in later time-windows (starting ~220ms) for both faces and squares. A similar dissociation was observed in gamma activity. Furthermore, whereas emotional responses habituated over time, the action-relevance effect increased during the course of the experiment, suggesting progressive learning based on the task needs. Our results support the hypothesis that the human amygdala mediates a broader relevance appraisal function, with the processing of emotion-relevance preceding temporally that of action-relevance

Emotional attention as a modulatory system of perception

Report for the scientific sojourn carried out at the University Medical Center, Swiss, from 2010 to 2012. Abundant evidence suggests that negative emotional stimuli are prioritized in the perceptual systems, eliciting enhanced neural responses in early sensory regions as compared with neutral information. This facilitated detection is generally paralleled by larger neural responses in early sensory areas, relative to the processing of neutral information. In this sense, the amygdala and other limbic regions, such as the orbitofrontal cortex, may play a critical role by sending modulatory projections onto the sensory cortices via direct or indirect feedback.The present project aimed at investigating two important issues regarding these mechanisms of emotional attention, by means of functional magnetic resonance imaging. In Study I, we examined the modulatory effects of visual emotion signals on the processing of task-irrelevant visual, auditory, and somatosensory input, that is, the intramodal and crossmodal effects of emotional attention. We observed that brain responses to auditory and tactile stimulation were enhanced during the processing of visual emotional stimuli, as compared to neutral, in bilateral primary auditory and somatosensory cortices, respectively. However, brain responses to visual task-irrelevant stimulation were diminished in left primary and secondary visual cortices in the same conditions. The results also suggested the existence of a multimodal network associated with emotional attention, presumably involving mediofrontal, temporal and orbitofrontal regions Finally, Study II examined the different brain responses along the low-level visual pathways and limbic regions, as a function of the number of retinal spikes during visual emotional processing. The experiment used stimuli resulting from an algorithm that simulates how the visual system perceives a visual input after a given number of retinal spikes. The results validated the visual model in human subjects and suggested differential emotional responses in the amygdala and visual regions as a function of spike-levels. A list of publications resulting from work in the host laboratory is included in the report

Emotional attention as a modulatory system of perception

Report for the scientific sojourn carried out at the University Medical Center, Swiss, from 2010 to 2012. Abundant evidence suggests that negative emotional stimuli are prioritized in the perceptual systems, eliciting enhanced neural responses in early sensory regions as compared with neutral information. This facilitated detection is generally paralleled by larger neural responses in early sensory areas, relative to the processing of neutral information. In this sense, the amygdala and other limbic regions, such as the orbitofrontal cortex, may play a critical role by sending modulatory projections onto the sensory cortices via direct or indirect feedback.The present project aimed at investigating two important issues regarding these mechanisms of emotional attention, by means of functional magnetic resonance imaging. In Study I, we examined the modulatory effects of visual emotion signals on the processing of task-irrelevant visual, auditory, and somatosensory input, that is, the intramodal and crossmodal effects of emotional attention. We observed that brain responses to auditory and tactile stimulation were enhanced during the processing of visual emotional stimuli, as compared to neutral, in bilateral primary auditory and somatosensory cortices, respectively. However, brain responses to visual task-irrelevant stimulation were diminished in left primary and secondary visual cortices in the same conditions. The results also suggested the existence of a multimodal network associated with emotional attention, presumably involving mediofrontal, temporal and orbitofrontal regions Finally, Study II examined the different brain responses along the low-level visual pathways and limbic regions, as a function of the number of retinal spikes during visual emotional processing. The experiment used stimuli resulting from an algorithm that simulates how the visual system perceives a visual input after a given number of retinal spikes. The results validated the visual model in human subjects and suggested differential emotional responses in the amygdala and visual regions as a function of spike-levels. A list of publications resulting from work in the host laboratory is included in the report

Gating of Auditory Novelty Processing by Emotional Context / Augment de processament de novetat pel context emocional

[eng] The present thesis encloses four studies that sought to asses the neural correlates, timing and modulatory effects of a negative emotional context on the processing of task-irrelevant novel sounds within the framework of auditory-visual oddball experimentation. Study I, II and III were conducted using event-related brain potentials (ERP) and Study IV used functional magnetic resonance imaging (fMRI). In Study I, fourteen healthy volunteers responded to a visual discrimination task, with either neutral or threatening sceneries, while a 64-channel electroencephalogram (EEG) was recorded. In Study II, fourteen healthy female volunteers responded to visual stimuli displaying either threatening or neutral sceneries, using an optimised version of the task, while a 64-channel electroencephalogram (EEG) was recorded. In Study III, fourteen female subjects and fourteen male subjects were recorded using a 64-channel electroencephalogram (EEG) while performing the same visual discrimination task as in Study II. In Study IV, seventeen healthy female volunteers responded to a visual colour discrimination task, with images of emotional facial expressions (angry, fearful or neutral), while neuroimaged in a 3Tesla scanner. In all experimental designs, single auditory stimuli, consisting of a majority of standard tones and infrequent novel environmental sounds, preceded the images and had to be ignored by the subjects. The main results in all four studies were pointing at comparable conclusions. Novel sounds elicited a distracting effect on subjects' performance, reflected by longer response times compared to those in standard trials. This effect was consistently magnified when preceding and following images were of a negative emotional load as compared to the neutral images. In Study III, women, but not men, showed this effect. Brain responses recorded with ERPs revealed, in Study I, an enhancement of late novelty-P3 responses to novel sounds in negative context, compared to the neutral one. Furthermore, Study II demonstrated that this modulatory effect can also occur in the early phase of this ERP component. Study III showed that the modulation of the early novelty-P3 was present only in women. Hemodynamic responses, in turn, showed that activation induced by novel sounds in superior temporal gyrus, comprising secondary cortex, planum temporale and primary auditory cortex, was enhanced when subjects responded to faces with a negative emotional expression compared to the neutral ones. The combination of results in the four studies show that the emotional context enhances excitability of auditory novelty cerebral regions at early stages of processing, making irrelevant sounds become more available in the attentional set under threatening conditions. Still, gender differences may be present in these effects, possibly due to differences in the evaluation of the emotional stimuli.[cat] La present tesi inclou quatre estudis que pretenen determinar els correlats neurals, temporalitat i efectes modulatoris d'un context emocional negatiu en el processament de sons novedosos irrellevants per la tasca en curs, en el marc de l'experimentació oddball auditivo- visual. L'Estudi I, II i III van ser duts a terme amb la tècnica de potencials evocats (PEs) i per l'Estudi IV es va utilitzar resonància magnètica funcional (RMf). A l'Estudi I, catorze voluntaris sans van respondre a una tasca de discriminació visual, amb escenes bé neutres, bé amenaçants, mentre s'enregistrava el seu electroencefalograma (EEG) de 64 canals. A l'Estudi II, catorze voluntàries sanes van respondre a estímuls visuals que representaven, bé escenes amenaçants o bé neutres, utilitzant una versió optimitzada de la tasca, mentre s'enregistrava el seu electroencefalograma (EEG) de 64 canals. A l'Estudi III, catorze dones i catorze homes van ser enregistrats utilitzant electroencefalograma (EEG) de 64 canals mentre realitzaven la mateixa tasca de discriminació visual que a l'Estudi II. A l'Estudi IV, disset voluntàries sanes van respondre a una tasca de discriminació de color, amb imatges d'expressions facials emocionals (d'amenaça, de por o neutra), mentre eren mesurades en un escàner de RMf de 3Tesla. En tots els dissenys experimentals, un estímul auditiu únic, que podia ser, bé en una majoria un to estàndard, o bé un so novedós ambiental infreqüent, precedia les imatges i havia d'ésser ignorat pels subjectes. Els principals resultats en tots quatre estudis apuntaren a conclusions comparables. Els sons novedosos produïren un efecte distractor en el rendiment dels subjectes, reflexat en temps de resposta més llargs comparats amb els assaigs amb sons estàndard. Aquest efecte va veure's consistentment magnificat quan les imatges precedents i següents tenien una càrrega emocional negativa en comparació a les imatges neutres. A l'Estudi III, les dones, però no els homes, van mostrar aquest efecte. Les respostes cerebrals enregistrades amb PEs revelaren, a l'Estudi I, un augment d'amplitud de la fase tardana del patró P3 de novetat davant de sons novedosos en context negatiu, en comparació amb el context neutre. A més, l'Estudi II va demostrar que aquest efecte modulatori pot ocórrer també en la fase primerenca d'aquest component de PE. L'Estudi III va mostrar que la modulació de la P3 de novetat era present només en dones. Les respostes hemodinàmiques, alhora, van mostrar que l'activació induïda pels sons novedosos al gir temporal superior, incloent escorça auditiva secundària, planum temporale i escorça auditiva primària, va ser augmentada quan els subjectes respongueren a cares amb una expressió emocional negativa en comparació amb les neutres. La combinació de resultats en els quatre estudis mostra que el context emocional augmenta l'excitabilitat de regions cerebrals en àrees primerenques de processament, fent que sons irrellevants es esdevinguin més disponibles en el set atencional en condicions d'amenaça. Tanmateix, és possible l'existència de diferències de gènere en aquests efectes, possiblement degut a diferències en l'avaluació dels estímuls emocionals. REFERENCES Alho, K., Winkler, I., Escera, C., Huotilainen, M., Virtanen, J., Jääskelainen, I.P., Pekkonen, E., Ilmoniemi, R.J. (1998). Processing of novel sounds and frequency changes in the human auditory cortex: magnetoencephalographic recordings. Psychophysiology, 35, 211-224. Bledowski, C., Prvulovic, D., Hoechstetter, K., Scherg, M., Wibral, M., Goebel, R., Linden, D.E.J. (2004). Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study. The Journal of Neuroscience, 24(42), 9353-9360. Bradley, M.M., Codispoti, M., Lang, P.J. (2006). A multi-process account of startle modulation during affective perception. Psychophysiology, 43, 486-497. Clark, V.P., Fannon, S., Lai, S., Benson, R., Bauer, L. (2000). Responses to rare visual target and distractor stimuli using event-related fMRI. Journal of Neurophysiology, 83, 3133-3139. Cuthbert, B.N., Schupp, H.T., Bradley, M.M., McManis, M., Lang, P.J. (1998). Probing affective pictures: attended startle and tone probes. Psychophysiology, 35, 344-347. Escera, C., Alho, K., Winkler, I., Näätäänen, R. (1998). Neural mechanisms of involuntary attention to acoustic novelty and change. Journal of Cognitive Neuroscience, 10(5), 590-604. Escera, C., Alho, K., Schröger, E., Winkler, I. (2000). Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiology and Neurootology, 5, 151-166

Impaired emotional biases in visual attention after bilateral amygdala lesion

It is debated whether the amygdala is critical for the emotional modulation of attention. While some studies show reduced attentional benefits for emotional stimuli in amygdala-damaged patients, others report preserved emotional effects. Various factors may account for these discrepant findings, including the temporal onset of the lesion, the completeness and severity of tissue damage, or the extent of neural plasticity and compensatory mechanisms, among others. Here, we investigated a rare patient with focal acute destruction of bilateral amygdala and adjacent hippocampal structures after late-onset herpetic encephalitis in adulthood. We compared her performance in two classic visual attention paradigms with that of healthy controls. First, we tested for any emotional advantage during an attentional blink task. Whereas controls showed better report of fearful and happy than neutral faces on trials with short lags between targets, the patient showed no emotional advantage, but also globally reduced report rates for all faces. Second, to ensure that memory disturbance due to hippocampal damage would not interfere with report performance, we also used a visual search task with either emotionally or visually salient face targets. Although the patient still exhibited efficient guided search for visually salient, non-emotional faces, her search slopes for emotional versus neutral faces showed no comparable benefit. In both tasks, however, changes in the patient predominated for happy more than fear stimuli, despite her normal explicit recognition of happy expressions. Our results provide new support for a causal role of the amygdala in emotional facilitation of visual attention, especially under conditions of increasing task-demands, and not limited to negative information. In addition, our data suggest that such deficits may not be amenable to plasticity and compensation, perhaps due to sudden and late-onset damage occurring in adulthood

Fear Spreading Across Senses: Visual Emotional Events Alter Cortical Responses to Touch, Audition, and Vision

Attention and perception are potentiated for emotionally significant stimuli, promoting efficient reactivity and survival. But does such enhancement extend to stimuli simultaneously presented across different sensory modalities? We used functional magnetic resonance imaging in humans to examine the effects of visual emotional signals on concomitant sensory inputs in auditory, somatosensory, and visual modalities. First, we identified sensory areas responsive to task-irrelevant tones, touches, or flickers, presented bilaterally while participants attended to either a neutral or a fearful face. Then, we measured whether these responses were modulated by the emotional content of the face. Sensory responses in primary cortices were enhanced for auditory and tactile stimuli when these appeared with fearful faces, compared with neutral, but striate cortex responses to the visual stimuli were reduced in the left hemisphere, plausibly as a consequence of sensory competition. Finally, conjunction and functional connectivity analyses identified 2 distinct networks presumably responsible for these emotional modulatory processes, involving cingulate, insular, and orbitofrontal cortices for the increased sensory responses, and ventrolateral prefrontal cortex for the decreased sensory responses. These results suggest that emotion tunes the excitability of sensory systems across multiple modalities simultaneously, allowing the individual to adaptively process incoming inputs in a potentially threatening environment

The perception of dynamic and static facial expressions of happiness and disgust investigated by ERPs and fMRI constrained source analysis.

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach