Multimodal Processing Of Emotional Meanings: A Hypothesis On The Adaptive Value Of Prosody

Humans combine multiple sources of information to comprehend meanings. These sources can be characterized as linguistic (i.e., lexical units and/or sentences) or paralinguistic (e.g. body posture, facial expression, voice intonation, pragmatic context). Emotion communication is a special case in which linguistic and paralinguistic dimensions can simultaneously denote the same, or multiple incongruous referential meanings. Think, for instance, about when someone says “I’m sad!”, but does so with happy intonation and a happy facial expression. Here, the communicative channels express very specific (although conflicting) emotional states as denotations. In such cases of intermodal incongruence, are we involuntarily biased to respond to information in one channel over the other? We hypothesize that humans are involuntary biased to respond to prosody over verbal content and facial expression, since the ability to communicate socially relevant information such as basic emotional states through prosodic modulation of the voice might have provided early hominins with an adaptive advantage that preceded the emergence of segmental speech (Darwin 1871; Mithen, 2005). To address this hypothesis, we examined the interaction between multiple communicative channels in recruiting attentional resources, within a Stroop interference task (i.e. a task in which different channels give conflicting information; Stroop, 1935). In experiment 1, we used synonyms of “happy” and “sad” spoken with happy and sad prosody. Participants were asked to identify the emotion expressed by the verbal content while ignoring prosody (Word task) or vice versa (Prosody task). Participants responded faster and more accurately in the Prosody task. Within the Word task, incongruent stimuli were responded to more slowly and less accurately than congruent stimuli. In experiment 2, we adopted synonyms of “happy” and “sad” spoken in happy and sad prosody, while a happy or sad face was displayed. Participants were asked to identify the emotion expressed by the verbal content while ignoring prosody and face (Word task), to identify the emotion expressed by prosody while ignoring verbal content and face (Prosody task), or to identify the emotion expressed by the face while ignoring prosody and verbal content (Face task). Participants responded faster in the Face task and less accurately when the two non-focused channels were expressing an emotion that was incongruent with the focused one, as compared with the condition where all the channels were congruent. In addition, in the Word task, accuracy was lower when prosody was incongruent to verbal content and face, as compared with the condition where all the channels were congruent. Our data suggest that prosody interferes with emotion word processing, eliciting automatic responses even when conflicting with both verbal content and facial expressions at the same time. In contrast, although processed significantly faster than prosody and verbal content, faces alone are not sufficient to interfere in emotion processing within a three-dimensional Stroop task. Our findings align with the hypothesis that the ability to communicate emotions through prosodic modulation of the voice – which seems to be dominant over verbal content - is evolutionary older than the emergence of segmental articulation (Mithen, 2005; Fitch, 2010). This hypothesis fits with quantitative data suggesting that prosody has a vital role in the perception of well-formed words (Johnson & Jusczyk, 2001), in the ability to map sounds to referential meanings (Filippi et al., 2014), and in syntactic disambiguation (Soderstrom et al., 2003). This research could complement studies on iconic communication within visual and auditory domains, providing new insights for models of language evolution. Further work aimed at how emotional cues from different modalities are simultaneously integrated will improve our understanding of how humans interpret multimodal emotional meanings in real life interactions

More than words (and faces): evidence for a Stroop effect of prosody in emotion word processing

Humans typically combine linguistic and nonlinguistic information to comprehend emotions. We adopted an emotion identification Stroop task to investigate how different channels interact in emotion communication. In experiment 1, synonyms of “happy” and “sad” were spoken with happy and sad prosody. Participants had more difficulty ignoring prosody than ignoring verbal content. In experiment 2, synonyms of “happy” and “sad” were spoken with happy and sad prosody, while happy or sad faces were displayed. Accuracy was lower when two channels expressed an emotion that was incongruent with the channel participants had to focus on, compared with the cross-channel congruence condition. When participants were required to focus on verbal content, accuracy was significantly lower also when prosody was incongruent with verbal content and face. This suggests that prosody biases emotional verbal content processing, even when conflicting with verbal content and face simultaneously. Implications for multimodal communication and language evolution studies are discussed

A Visual Pathway Links Brain Structures Active during Magnetic Compass Orientation in Migratory Birds

The magnetic compass of migratory birds has been suggested to be light-dependent. Retinal cryptochrome-expressing neurons and a forebrain region, “Cluster N”, show high neuronal activity when night-migratory songbirds perform magnetic compass orientation. By combining neuronal tracing with behavioral experiments leading to sensory-driven gene expression of the neuronal activity marker ZENK during magnetic compass orientation, we demonstrate a functional neuronal connection between the retinal neurons and Cluster N via the visual thalamus. Thus, the two areas of the central nervous system being most active during magnetic compass orientation are part of an ascending visual processing stream, the thalamofugal pathway. Furthermore, Cluster N seems to be a specialized part of the visual wulst. These findings strongly support the hypothesis that migratory birds use their visual system to perceive the reference compass direction of the geomagnetic field and that migratory birds “see” the reference compass direction provided by the geomagnetic field

Humans Recognize Vocal Expressions Of Emotional States Universally Across Species

The perception of danger in the environment can induce physiological responses (such as a heightened state of arousal) in animals, which may cause measurable changes in the prosodic modulation of the voice (Briefer, 2012). The ability to interpret the prosodic features of animal calls as an indicator of emotional arousal may have provided the first hominins with an adaptive advantage, enabling, for instance, the recognition of a threat in the surroundings. This ability might have paved the ability to process meaningful prosodic modulations in the emerging linguistic utterances

When One Hemisphere Takes Control: Metacontrol in Pigeons (Columba livia)

Vertebrate brains are composed of two hemispheres that receive input, compute, and interact to form a unified response. How the partially different processes of both hemispheres are integrated to create a single output is largely unknown. In some cases one hemisphere takes charge of the response selection--a process known as metacontrol. Thus far, this phenomenon has only been shown in a handful of studies with primates, mostly conducted in humans. Metacontrol, however, is even more relevant for animals like birds with laterally placed eyes and complete chiasmatic decussation since visual input to the hemispheres is largely different.Homing pigeons (Columba livia) were trained with a color discrimination task. Each hemisphere was trained with a different color pair and therefore had a different experience. Subsequently, the pigeons were binocularly examined with two additional stimuli that combined the positive color of one hemisphere with a negative color that had been shown to the other, omitting the availability of a coherent solution and confronting the pigeons with a conflicting situation. Some of the pigeons responded to both stimuli, indicating that none of the hemispheres dominated the overall preference. Some birds, however, responded primarily to one of the conflicting stimuli, showing that they based their choice on the left- or right-monocularly learned color pair, indicating hemispheric metacontrol.We could demonstrate for the first time that metacontrol is a widespread phenomenon that also exists in birds, and thus in principle requires no corpus callosum. Our results are closely similar to those in humans: monocular performance was higher than binocular one and animals displayed different modes of hemispheric dominance. Thus, metacontrol is a dynamic and widely distributed process that possibly constitutes a requirement for all animals with a bipartite brain to confront the problem of choosing between two hemisphere-bound behavioral options

Vesicular stomatitis virus enables gene transfer and transsynaptic tracing in a wide range of organisms

Current limitations in technology have prevented an extensive analysis of the connections among neurons, particularly within nonmammalian organisms. We developed a transsynaptic viral tracer originally for use in mice, and then tested its utility in a broader range of organisms. By engineering the vesicular stomatitis virus (VSV) to encode a fluorophore and either the rabies virus glycoprotein (RABV-G) or its own glycoprotein (VSV-G), we created viruses that can transsynaptically label neuronal circuits in either the retrograde or anterograde direction, respectively. The vectors were investigated for their utility as polysynaptic tracers of chicken and zebrafish visual pathways. They showed patterns of connectivity consistent with previously characterized visual system connections, and revealed several potentially novel connections. Further, these vectors were shown to infect neurons in several other vertebrates, including Old and New World monkeys, seahorses, axolotls, and Xenopus. They were also shown to infect two invertebrates, Drosophila melanogaster, and the box jellyfish, Tripedalia cystophora, a species previously intractable for gene transfer, although no clear evidence of transsynaptic spread was observed in these species. These vectors provide a starting point for transsynaptic tracing in most vertebrates, and are also excellent candidates for gene transfer in organisms that have been refractory to other methods

Mirror-Mark Tests Performed on Jackdaws Reveal Potential Methodological Problems in the Use of Stickers in Avian Mark-Test Studies

Some animals are capable of recognizing themselves in a mirror, which is considered to be demonstrated by passing the mark test. Mirror self-recognition capacity has been found in just a few mammals having very large brains and only in one bird, the magpie (Pica pica). The results obtained in magpies have enormous biological and cognitive implications because the fact that magpies were able to pass the mark test meant that this species is at the same cognitive level with great apes, that mirror self-recognition has evolved independently in the magpie and great apes (which diverged 300 million years ago), and that the neocortex (which is not present in the bird's brains) is not a prerequisite for mirror self-recognition as previously believed. Here, we have replicated the experimental design used on magpies to determine whether jackdaws (Corvus monedula) are also capable of mirror self-recognition by passing the mark test. We found that our nine jackdaws showed a very high interest towards the mirror and exhibited self-contingent behavior as soon as mirrors were introduced. However, jackdaws were not able to pass the mark test: both sticker-directed actions and sticker removal were performed with a similar frequency in both the cardboard (control) and the mirror conditions. We conclude that our jackdaws' behaviour raises non-trivial questions about the methodology used in the avian mark test. Our study suggests that the use of self-adhesive stickers on sensitive throat feathers may open the way to artefactual results because birds might perceive the stickers tactilely.JMPS was funded by Ministerio de Educación and Consejería de Innovación, C 420 iencia y Empresa under International Excellence Campus Program (CEI Granada) and TPC was funded by Ministerio de Educación y Ciencia by a postdoctoral contract from the project CGL2011-25634

Plasticity in D1-Like Receptor Expression Is Associated with Different Components of Cognitive Processes

Dopamine D1-like receptors consist of D1 (D1A) and D5 (D1B) receptors and play a key role in working memory. However, their possibly differential contribution to working memory is unclear. We combined a working memory training protocol with a stepwise increase of cognitive subcomponents and real-time RT-PCR analysis of dopamine receptor expression in pigeons to identify molecular changes that accompany training of isolated cognitive subfunctions. In birds, the D1-like receptor family is extended and consists of the D1A, D1B, and D1D receptors. Our data show that D1B receptor plasticity follows a training that includes active mental maintenance of information, whereas D1A and D1D receptor plasticity in addition accompanies learning of stimulus-response associations. Plasticity of D1-like receptors plays no role for processes like response selection and stimulus discrimination. None of the tasks altered D2 receptor expression. Our study shows that different cognitive components of working memory training have distinguishable effects on D1-like receptor expression

No evidence that footedness in pheasants influences cognitive performance in tasks assessing colour discrimination and spatial ability

The differential specialization of each side of the brain facilitates the parallel processing of information and has been documented in a wide range of animals. Animals that are more lateralized as indicated by consistent preferential limb use are commonly reported to exhibit superior cognitive ability as well as other behavioural advantages.We assayed the lateralization of 135 young pheasants (Phasianus colchicus), indicated by their footedness in a spontaneous stepping task, and related this measure to individual performance in either 3 assays of visual or spatial learning and memory. We found no evidence that pronounced footedness enhances cognitive ability in any of the tasks. We also found no evidence that an intermediate footedness relates to better cognitive performance. This lack of relationship is surprising because previous work revealed that pheasants have a slight population bias towards right footedness, and when released into the wild, individuals with higher degrees of footedness were more likely to die. One explanation for why extreme lateralization is constrained was that it led to poorer cognitive performance, or that optimal cognitive performance was associated with some intermediate level of lateralization. This stabilizing selection could explain the pattern of moderate lateralization that is seen in most non-human species that have been studied. However, we found no evidence in this study to support this explanation