Preliminary Evidence of Pre-Attentive Distinctions of Frequency-Modulated Tones that Convey Affect

Recognizing emotion is an evolutionary imperative. An early stage of auditory scene analysis involves the perceptual grouping of acoustic features, which can be based on both temporal coincidence and spectral features such as perceived pitch. Perceived pitch, or fundamental frequency (F0), is an especially salient cue for differentiating affective intent through speech intonation (prosody). We hypothesized that: (1) simple frequency-modulated tone abstractions, based on the parameters of actual prosodic stimuli, would be reliably classified as representing differing emotional categories; and (2) that such differences would yield significant mismatch negativities (MMNs) – an index of pre-attentive deviance detection within the auditory environment. We constructed a set of FM tones that approximated the F0 mean and variation of reliably recognized happy and neutral prosodic stimuli. These stimuli were presented to 13 subjects using a passive listening oddball paradigm. We additionally included stimuli with no frequency modulation (FM) and FM tones with identical carrier frequencies but differing modulation depths as control conditions. Following electrophysiological recording, subjects were asked to identify the sounds they heard as happy, sad, angry, or neutral. We observed that FM tones abstracted from happy and no-expression speech stimuli elicited MMNs. Post hoc behavioral testing revealed that subjects reliably identified the FM tones in a consistent manner. Finally, we also observed that FM tones and no-FM tones elicited equivalent MMNs. MMNs to FM tones that differentiate affect suggests that these abstractions may be sufficient to characterize prosodic distinctions, and that these distinctions can be represented in pre-attentive auditory sensory memory

Comment:The Next Frontier: Prosody Research Gets Interpersonal

Neurocognitive models (e.g., Schirmer & Kotz, 2006) have helped to characterize how listeners incrementally derive meaning from vocal expressions of emotion in spoken language, what neural mechanisms are involved at different processing stages, and their relative time course. But how can these insights be applied to communicative situations in which prosody serves a predominantly interpersonal function? This comment examines recent data highlighting the dynamic interplay of prosody and language, when vocal attributes serve the sociopragmatic goals of the speaker or reveal interpersonal information that listeners use to construct a mental representation of what is being communicated. Our comment serves as a beacon to researchers interested in how the neurocognitive system “makes sense” of socioemotive aspects of prosody

High emotional contagion and empathy are associated with enhanced detection of emotional authenticity in laughter

Nonverbal vocalisations such as laughter pervade social interactions, and the ability to accurately interpret them is an important skill. Previous research has probed the general mechanisms supporting vocal emotional processing, but the factors that determine individual differences in this ability remain poorly understood. Here, we ask whether the propensity to resonate with others’ emotions—as measured by trait levels of emotional contagion and empathy—relates to the ability to perceive different types of laughter. We focus on emotional authenticity detection in spontaneous and voluntary laughs: spontaneous laughs reflect a less controlled and genuinely felt emotion, and voluntary laughs reflect a more deliberate communicative act (e.g., polite agreement). In total, 119 participants evaluated the authenticity and contagiousness of spontaneous and voluntary laughs and completed two self-report measures of resonance with others’ emotions: the Emotional Contagion Scale and the Empathic Concern scale of the Interpersonal Reactivity Index. We found that higher scores on these measures predict enhanced ability to detect laughter authenticity. We further observed that perceived contagion responses during listening to laughter significantly relate to authenticity detection. These findings suggest that resonating with others’ emotions provides a mechanism for processing complex aspects of vocal emotional information.info:eu-repo/semantics/acceptedVersio

The paradoxical role of emotional intensity in the perception of vocal affect

Vocalizations including laughter, cries, moans, or screams constitute a potent source of information about the affective states of others. It is typically conjectured that the higher the intensity of the expressed emotion, the better the classification of affective information. However, attempts to map the relation between affective intensity and inferred meaning are controversial. Based on a newly developed stimulus database of carefully validated non-speech expressions ranging across the entire intensity spectrum from low to peak, we show that the intuition is false. Based on three experiments (N = 90), we demonstrate that intensity in fact has a paradoxical role. Participants were asked to rate and classify the authenticity, intensity and emotion, as well as valence and arousal of the wide range of vocalizations. Listeners are clearly able to infer expressed intensity and arousal; in contrast, and surprisingly, emotion category and valence have a perceptual sweet spot: moderate and strong emotions are clearly categorized, but peak emotions are maximally ambiguous. This finding, which converges with related observations from visual experiments, raises interesting theoretical challenges for the emotion communication literature

The role of attention in the processing of emotional vocalizations: ERP insights

Dissertação de mestrado integrado em Psicologia, (área de especialização em Psicologia Clínica e da Saúde)Identificar rapidamente as emoções transmitidas pela voz e pelas faces dos outros é fundamental para um funcionamento social adequado. Vários estudos comportamentais e electrofisiológicos analisaram o processamento dos sinais usados na comunicação emocional. Contudo, a maioria utilizou faces ou palavras faladas como estímulos. Assim, ainda pouco se sabe sobre os correlatos neuronais e o desenvolvimento temporal do processamento de vocalizações não-verbais. O presente estudo usou a técnica de ERP (potenciais relacionados com eventos) para estudar o processamento de vocalizações não-verbais emocionais (alegria e raiva) e neutras num estado tardio do processamento. O componente de onda P300 foi analisado. Foi descoberto um efeito modulatório da emoção sobre a amplitude deste. Vocalizações de alegria e raiva evocaram amplitudes mais positivas para o componente P300 em comparação com neutras. Mais, foi encontrado um efeito do contexto emocional no processamento de sons neutros. Vocalizações de neutralidade num contexto de raiva evocaram um componente P300 de amplitude mais positiva em comparação com vocalizações de neutralidade num contexto de alegria. Adicionalmente, foi observada uma diferença de género: verificou-se uma amplitude mais positiva do P300 para mulheres. Estes resultados sugerem um efeito da valência dos estímulos a um nível atencional e de memória imediata, indexado pelo componente P300.Rapidly and effectively identifying the emotions conveyed by others’ faces and voices is fundamental for an adequate social functioning. Several behavioral and electrophysiological studies have analyzed the processing of emotional communication signals. Nonetheless, the majority used faces or spoken words as stimuli. Yet, little is known about the neural correlates and temporal course of the processing of non-verbal vocalizations. The present study used the ERP (Event-Related Potential) methodology to study the processing of non-verbal emotional (angry and happy) versus neutral vocalizations at a later stage of processing. The P300 component was analyzed. Its amplitude was differently modulated as a function of emotion. Angry and happy vocalizations elicited more positive amplitudes for the P300 component in comparison with neutral ones. Furthermore, emotional context was found to have an effect on the processing of neutral sounds. Neutral vocalizations in an angry context elicited a more positive amplitude P300, in comparison to neutral vocalizations in a happy context. Furthermore, a gender difference was observed: the P300 amplitude was found to be more positive for female relative to male participants. Together, these findings suggest an effect of stimulus (non-verbal vocalizations) valence at an attentional and immediate memory level, as indexed by the P300 component

How Bodies and Voices Interact in Early Emotion Perception

Successful social communication draws strongly on the correct interpretation of others' body and vocal expressions. Both can provide emotional information and often occur simultaneously. Yet their interplay has hardly been studied. Using electroencephalography, we investigated the temporal development underlying their neural interaction in auditory and visual perception. In particular, we tested whether this interaction qualifies as true integration following multisensory integration principles such as inverse effectiveness. Emotional vocalizations were embedded in either low or high levels of noise and presented with or without video clips of matching emotional body expressions. In both, high and low noise conditions, a reduction in auditory N100 amplitude was observed for audiovisual stimuli. However, only under high noise, the N100 peaked earlier in the audiovisual than the auditory condition, suggesting facilitatory effects as predicted by the inverse effectiveness principle. Similarly, we observed earlier N100 peaks in response to emotional compared to neutral audiovisual stimuli. This was not the case in the unimodal auditory condition. Furthermore, suppression of beta–band oscillations (15–25 Hz) primarily reflecting biological motion perception was modulated 200–400 ms after the vocalization. While larger differences in suppression between audiovisual and audio stimuli in high compared to low noise levels were found for emotional stimuli, no such difference was observed for neutral stimuli. This observation is in accordance with the inverse effectiveness principle and suggests a modulation of integration by emotional content. Overall, results show that ecologically valid, complex stimuli such as joined body and vocal expressions are effectively integrated very early in processing

The automatic processing of non-verbal emotional vocalizations: an electrophysiological investigation

Dissertação de mestrado integrado em Psicologia (área de especialização em Psicologia Clínica e da Saúde)The human voice is a critical channel for the exchange of information about the emotionality of a speaker. In this sense, it is important to investigate the neural correlates of non-verbal vocalizations processing, even when listeners are not attending to these events. We developed an oddball paradigm in which emotional (happy and angry) and neutral vocalizations were presented both as standard and deviant stimuli in four conditions: Happy, Angry, Neutral 1 (neutral vocalizations with angry context), and Neutral 2 (neutral vocalizations with happy context). To unfold the time course of the auditory change detection mechanisms indexed by the Mismatch Negativity (MMN) component, the Event–Related Potentials (ERP) methodology was used. ERPs were recorded in 17 healthy subjects. The results showed that Happy and Neutral 2 conditions elicited more negative MMN amplitude relative to the Angry condition, at midline (Fz, Cz) electrodes. Overall results suggest that automatic auditory change detection is enhanced for positive and neutral (in happy context) vocalizations than for negative stimuli.A voz humana é um canal vital na troca de informação sobre a emocionalidade do outro. Neste sentido, é importante investigar quais os correlatos neuronais associados ao processamento de vocalizações não-verbais, mesmo quando não é alocada atenção a estes estímulos. Foi criado um paradigma oddball com vocalizações emocionais (alegria e raiva) e neutras, que eram apresentadas como estímulos frequentes ou infrequentes em quatro condições distintas: Alegre, Raiva, Neutro 1 (vocalizações neutras em contexto de raiva) e Neutro 2 (vocalizações neutras em contexto de alegria). Para investigar o curso temporal dos mecanismos automáticos de detecção de mudança auditiva, foi usada a técnica de Potenciais Evocados e estudado o componente Mismatch Negativity (MMN). A amostra foi constituída por 17 indivíduos saudáveis. Os resultados mostraram que as condições Alegre e Neutro 2 elicitaram uma amplitude de MMN mais negativa comparativamente com a condição Raiva, para os eléctrodos situados na linha média do escalpe (Fz, Cz). Estes resultados indicam que existe um mecanismo neuronal de deteção de mudança auditiva mais pronunciado para vocalizações positivas e neutras (em contexto de alegria) comparativamente com vocalizações negativas

Emotional Speech Perception Unfolding in Time: The Role of the Basal Ganglia

The basal ganglia (BG) have repeatedly been linked to emotional speech processing in studies involving patients with neurodegenerative and structural changes of the BG. However, the majority of previous studies did not consider that (i) emotional speech processing entails multiple processing steps, and the possibility that (ii) the BG may engage in one rather than the other of these processing steps. In the present study we investigate three different stages of emotional speech processing (emotional salience detection, meaning-related processing, and identification) in the same patient group to verify whether lesions to the BG affect these stages in a qualitatively different manner. Specifically, we explore early implicit emotional speech processing (probe verification) in an ERP experiment followed by an explicit behavioral emotional recognition task. In both experiments, participants listened to emotional sentences expressing one of four emotions (anger, fear, disgust, happiness) or neutral sentences. In line with previous evidence patients and healthy controls show differentiation of emotional and neutral sentences in the P200 component (emotional salience detection) and a following negative-going brain wave (meaning-related processing). However, the behavioral recognition (identification stage) of emotional sentences was impaired in BG patients, but not in healthy controls. The current data provide further support that the BG are involved in late, explicit rather than early emotional speech processing stages

Emotional Cues during Simultaneous Face and Voice Processing: Electrophysiological Insights

Both facial expression and tone of voice represent key signals of emotional communication but their brain processing correlates remain unclear. Accordingly, we constructed a novel implicit emotion recognition task consisting of simultaneously presented human faces and voices with neutral, happy, and angry valence, within the context of recognizing monkey faces and voices task. To investigate the temporal unfolding of the processing of affective information from human face-voice pairings, we recorded event-related potentials (ERPs) to these audiovisual test stimuli in 18 normal healthy subjects; N100, P200, N250, P300 components were observed at electrodes in the frontal-central region, while P100, N170, P270 were observed at electrodes in the parietal-occipital region. Results indicated a significant audiovisual stimulus effect on the amplitudes and latencies of components in frontal-central (P200, P300, and N250) but not the parietal occipital region (P100, N170 and P270). Specifically, P200 and P300 amplitudes were more positive for emotional relative to neutral audiovisual stimuli, irrespective of valence, whereas N250 amplitude was more negative for neutral relative to emotional stimuli. No differentiation was observed between angry and happy conditions. The results suggest that the general effect of emotion on audiovisual processing can emerge as early as 200 msec (P200 peak latency) post stimulus onset, in spite of implicit affective processing task demands, and that such effect is mainly distributed in the frontal-central region

Seeing Emotion with Your Ears: Emotional Prosody Implicitly Guides Visual Attention to Faces

Interpersonal communication involves the processing of multimodal emotional cues, particularly facial expressions (visual modality) and emotional speech prosody (auditory modality) which can interact during information processing. Here, we investigated whether the implicit processing of emotional prosody systematically influences gaze behavior to facial expressions of emotion. We analyzed the eye movements of 31 participants as they scanned a visual array of four emotional faces portraying fear, anger, happiness, and neutrality, while listening to an emotionally-inflected pseudo-utterance (Someone migged the pazing) uttered in a congruent or incongruent tone. Participants heard the emotional utterance during the first 1250 milliseconds of a five-second visual array and then performed an immediate recall decision about the face they had just seen. The frequency and duration of first saccades and of total looks in three temporal windows ([0–1250 ms], [1250–2500 ms], [2500–5000 ms]) were analyzed according to the emotional content of faces and voices. Results showed that participants looked longer and more frequently at faces that matched the prosody in all three time windows (emotion congruency effect), although this effect was often emotion-specific (with greatest effects for fear). Effects of prosody on visual attention to faces persisted over time and could be detected long after the auditory information was no longer present. These data imply that emotional prosody is processed automatically during communication and that these cues play a critical role in how humans respond to related visual cues in the environment, such as facial expressions