Investigating the Dynamics of Nonverbal Communication: Evidence from Neuroimaging in High-Functioning Autism and Typical Development

Being able to accurately decode nonverbal behaviors is essential for human communication. However, meaningful information is not only conveyed by static cues, which have already been extensively investigated in the past, but also by subtle movement dynamics. Investigating such subtle movement dynamics opens up new avenues for the investigation of social cognition in healthy and psychopathological contexts. Thus, this thesis investigated the correlates of perceiving and processing dynamic nonverbal social cues in the human brain using fMRI along with behavioral methods. First, we demonstrate the utility and validity of several digital simulation paradigms for nonverbal behavior research in a neuroimaging context. Further, we show for the first time that the observation of nonverbal communicative interactions is modulated by different spatiotemporal factors of perceived movement and that this is associated with both the AON and the SNN. In addition, we contribute to an ongoing debate by showing that in the context of the observation of complex human interactions, there is no biological bias of the AON. Furthermore, we demonstrate for the first time the domain-specificity of the two neural networks AON and SNN in the context of observing complex human interactions. Finally, we demonstrate that HFA is associated with deficits in social processing more so than social perception, but that social perception may rely on other cognitive strategies compared to typically developing individuals

Spring School on Language, Music, and Cognition: Organizing Events in Time

The interdisciplinary spring school “Language, music, and cognition: Organizing events in time” was held from February 26 to March 2, 2018 at the Institute of Musicology of the University of Cologne. Language, speech, and music as events in time were explored from different perspectives including evolutionary biology, social cognition, developmental psychology, cognitive neuroscience of speech, language, and communication, as well as computational and biological approaches to language and music. There were 10 lectures, 4 workshops, and 1 student poster session. Overall, the spring school investigated language and music as neurocognitive systems and focused on a mechanistic approach exploring the neural substrates underlying musical, linguistic, social, and emotional processes and behaviors. In particular, researchers approached questions concerning cognitive processes, computational procedures, and neural mechanisms underlying the temporal organization of language and music, mainly from two perspectives: one was concerned with syntax or structural representations of language and music as neurocognitive systems (i.e., an intrapersonal perspective), while the other emphasized social interaction and emotions in their communicative function (i.e., an interpersonal perspective). The spring school not only acted as a platform for knowledge transfer and exchange but also generated a number of important research questions as challenges for future investigations

Spontaneous facial mimicry is modulated by joint attention and autistic traits

Joint attention (JA) and spontaneous facial mimicry (SFM) are fundamental processes in social interactions, and they are closely related to empathic abilities. When tested independently, both of these processes have been usually observed to be atypical in individuals with autism spectrum conditions (ASC). However, it is not known how these processes interact with each other in relation to autistic traits. This study addresses this question by testing the impact of JA on SFM of happy faces using a truly interactive paradigm. Sixty-two neurotypical participants engaged in gaze-based social interaction with an anthropomorphic, gaze-contingent virtual agent. The agent either established JA by initiating eye contact or looked away, before looking at an object and expressing happiness or disgust. Eye tracking was used to make the agent's gaze behavior and facial actions contingent to the participants' gaze. SFM of happy expressions was measured by Electromyography (EMG) recording over the Zygomaticus Major muscle. Results showed that JA augments SFM in individuals with low compared with high autistic traits. These findings are in line with reports of reduced impact of JA on action imitation in individuals with ASC. Moreover, they suggest that investigating atypical interactions between empathic processes, instead of testing these processes individually, might be crucial to understanding the nature of social deficits in autis

Serotonin and Social Competency

Despite the fact that serotonergic drugs are called upon to treat a myriad of psychopathologies, the effect of serotonin on core behaviors and cognitive abilities are poorly understood. This is especially true for cognitive functions which underlie socially competent behavior. This dissertation aims to increase understanding of the role of serotonin in other monitoring and social competency throughout development. Species of primates, including humans, that live in complex social environments must allocate extensive cognitive resources to monitor conspecifics. However, they must balance the benefits of gathering social information with the need to monitor their non-social environments. Social monitoring strategies vary across species, populations of the same species, and even within populations. This variation seems to be dependent upon the amount of social monitoring that is required for an individual to avoid conflict and maintain its dominance rank. The serotonergic system has a history connecting it to socially competent behavior, like other monitoring, although its causal role is not understood. Therefore, better understanding how increasing concentrations of serotonin impact other monitoring behaviors will clarify serotonin’s role in psychopathology and may help clinicians predict how serotonergic interventions will influence pathologies. Furthermore, better understanding how the relationship between early life stress and serotonin impacts social competency will improve our understanding of psychiatric disorders and help develop novel interventions.In Chapter 2 of the present dissertation, the role of serotonin in the allocation of attention to social images, a core component of social monitoring, was studied by assaying rhesus macaques’ unconstrained looking to social and non-social stimuli using a free viewing paradigm (Dal Monte et al., 2014). We used a quantitative, repeated, within-subject, design to test how increasing central concentrations of serotonin would impact social looking behavior. Importantly, we found that increasing central concentrations of serotonin with its direct precursor, 5-Hydroxytrypotophan (5-HTP), modulated looking duration relative to individual differences in looking. 5-HTP decreased looking duration in animals with high baseline attention, but increased looking duration in low baseline attention animals. 5-HTP’s effects were also reflected in how engaged individuals were in the task and how they allocated attention to salient facial features—the eyes and mouth—of stimulus animals. Individual differences seem to be based in serotonergic function. Compared to low baseline animals, high baseline looking animals exhibited higher baseline concentrations of 5-HTP and serotonin and lower 5-HIAA to serotonin ratios indicating central serotonergic functioning may underlie and predict variation in serotonin’s effects on cognitive operation. The individual differences in 5-HTPs effects on looking increased our interest in serotonin’s role in balancing the costs and benefits of monitoring others (Weinberg‐Wolf and Chang, 2019). In Chapter 3, we tested the effect of 5-HTP on macaque’s abilities to flexibly switch between two actions: orienting to faces, or, at other times, inhibiting orientation towards faces. Critically, we found that 5-HTP also only impaired the ability to inhibit orientation to faces, but did not impact inhibition performance on trials with control images. It also seems that 5-HTP made animals less flexible, causing them to persevere in actions more. Furthermore, 5-HTP’s effects on performance are likely due to changes in arousal and motivation state as 5-HTP’s impairments were linked to increased reaction time, animals taking longer to initiate trials, and a constricted pupil. Serotonin is also implicated in the development of psychiatric disorders, especially Autism Spectrum Disorders. In Chapter 4, we examined the relationship between infant serotonergic function, assayed via CSF concentrations of 5-HIAA, and the acquisition of social status. We found that neonatal (11-32 days) 5-HIAA concentrations positively predict eventual, acquired, social rank. Furthermore, this relationship was strongest amongst macaques who had been reared by their mothers compared to those reared without mothers. In addition, mother reared infants exhibited higher concentrations of CSF 5-HIAA and attained higher social rank than their peers. These finds support the relationship between serotonin and early social experience in socially competent development. By considering the findings presented in Chapters 2, 3 and 4, we discuss, in Chapter 5, the role of serotonin in competent social monitoring and its development. We also discuss a plausible evolutionary explanation for variability in other monitoring behaviors and for variable effects of serotonin on cognition and behavior. Finally, we consider future directions researchers should explore as the field progresses

The Role of Eye Gaze During Natural Social Interactions in Typical and Autistic People

Social interactions involve complex exchanges of a variety of social signals, such as gaze, facial expressions, speech and gestures. Focusing on the dual function of eye gaze, this review explores how the presence of an audience, communicative purpose and temporal dynamics of gaze allow interacting partners to achieve successful communication. First, we focus on how being watched modulates social cognition and behavior. We then show that the study of interpersonal gaze processing, particularly gaze temporal dynamics, can provide valuable understanding of social behavior in real interactions. We propose that the Interpersonal Gaze Processing model, which combines both sensing and signaling functions of eye gaze, provides a framework to make sense of gaze patterns in live interactions. Finally, we discuss how autistic individuals process the belief in being watched and interpersonal dynamics of gaze, and suggest that systematic manipulation of factors modulating gaze signaling can reveal which aspects of social eye gaze are challenging in autism

Integration of Facial Expression and Gaze Direction in Individuals with a High Level of Autistic Traits

This work was supported by the Spanish Ministry of Economy, Industry, and Competitiveness, through research project PID2020-114790GB-I00 to J.L., and by the Andalusian Council and European Regional Development Fund, through research project B-SEJ-572-UGR20 to A.M. B.A.-M. was supported by a predoctoral fellowship from the FPU program from the Spanish Ministry of Science and Education (FPU16/07124).Background.We investigated whether individuals with high levels of autistic traits integrate relevant communicative signals, such as facial expression, when decoding eye-gaze direction. Methods. Students with high vs. low scores on the Autism Spectrum Quotient (AQ) performed a task in which they responded to the eye directions of faces, presented on the left or the right side of a screen, portraying different emotional expressions. Results. In both groups, the identification of gaze direction was faster when the eyes were directed towards the center of the scene. However, in the low AQ group, this effect was larger for happy faces than for neutral faces or faces showing other emotional expressions, whereas participants from high AQ group were not affected by emotional expressions. Conclusions. These results suggest that individuals with more autistic traits may not integrate multiple communicative signals based on their emotional value.Spanish Government PID2020-114790GB-I00Andalusian CouncilEuropean Commission B-SEJ-572-UGR20German Research Foundation (DFG) FPU16/0712

Using dual eye tracking to uncover personal gaze patterns during social interaction

We report the personal eye gaze patterns of people engaged in face-to-face getting acquainted conversation. Considerable differences between individuals are underscored by a stability of eye gaze patterns within individuals. Results suggest the existence of an eye-mouth gaze continuum. This continuum includes some people showing a strong preference for eye gaze, some with a strong preference for mouth gaze, and others distributing their gaze between the eyes and mouth to varying extents. Additionally, we found evidence of within-participant consistency not just for location preference but also for the duration of fixations upon the eye and mouth regions. We also estimate that during a 4-minute getting acquainted conversation mutual face gaze constitutes about 60% of conversation that occurs via typically brief instances of 2.2 seconds. Mutual eye contact ranged from 0-45% of conversation, via very brief instances. This was despite participants subjectively perceiving eye contact occurring for about 70% of conversation. We argue that the subjective perception of eye contact is a product of mutual face gaze instead of actual mutual eye contact. We also outline the fast activity of gaze movements upon various locations both on and off face during a typical face-to-face conversation

Social signalling as a framework for second-person neuroscience

Despite the recent increase in second-person neuroscience research, it is still hard to understand which neurocognitive mechanisms underlie real-time social behaviours. Here, we propose that social signalling can help us understand social interactions both at the single- and two-brain level in terms of social signal exchanges between senders and receivers. First, we show how subtle manipulations of being watched provide an important tool to dissect meaningful social signals. We then focus on how social signalling can help us build testable hypotheses for second-person neuroscience with the example of imitation and gaze behaviour. Finally, we suggest that linking neural activity to specific social signals will be key to fully understand the neurocognitive systems engaged during face-to-face interactions