A uniform human multimodal dataset for emotion perception and judgment

Face perception is a fundamental aspect of human social interaction, yet most research on this topic has focused on single modalities and specific aspects of face perception. Here, we present a comprehensive multimodal dataset for examining facial emotion perception and judgment. This dataset includes EEG data from 97 unique neurotypical participants across 8 experiments, fMRI data from 19 neurotypical participants, single-neuron data from 16 neurosurgical patients (22 sessions), eye tracking data from 24 neurotypical participants, behavioral and eye tracking data from 18 participants with ASD and 15 matched controls, and behavioral data from 3 rare patients with focal bilateral amygdala lesions. Notably, participants from all modalities performed the same task. Overall, this multimodal dataset provides a comprehensive exploration of facial emotion perception, emphasizing the importance of integrating multiple modalities to gain a holistic understanding of this complex cognitive process. This dataset serves as a key missing link between human neuroimaging and neurophysiology literature, and facilitates the study of neuropsychiatric populations

The relation between neuronal firing, local field potentials and hemodynamic activity in the human amygdala in response to aversive dynamic visual stimuli

The amygdala is a central part of networks of brain regions underlying perception and cognition, in particular related to processing of emotionally salient stimuli. Invasive electrophysiological and hemodynamic measurements are commonly used to evaluate functions of the human amygdala, but a comprehensive understanding of their relation is still lacking. Here, we aimed at investigating the link between fast and slow frequency amygdalar oscillations, neuronal firing and hemodynamic responses. To this aim, we recorded intracranial electroencephalography (iEEG), hemodynamic responses and single neuron activity from the amygdala of patients with epilepsy. Patients were presented with dynamic visual sequences of fearful faces (aversive condition), interleaved with sequences of neutral landscapes (neutral condition). Comparing responses to aversive versus neutral stimuli across participants, we observed enhanced high gamma power (HGP, >60 ​Hz) during the first 2 ​s of aversive sequence viewing, and reduced delta power (1-4 ​Hz) lasting up to 18 ​s. In 5 participants with implanted microwires, neuronal firing rates were enhanced following aversive stimuli, and exhibited positive correlation with HGP and hemodynamic responses. Our results show that high gamma power, neuronal firing and BOLD responses from the human amygdala are co-modulated. Our findings provide, for the first time, a comprehensive investigation of amygdalar responses to aversive stimuli, ranging from single-neuron spikes to local field potentials and hemodynamic responses