The perception of emotion in artificial agents

Given recent technological developments in robotics, artificial intelligence and virtual reality, it is perhaps unsurprising that the arrival of emotionally expressive and reactive artificial agents is imminent. However, if such agents are to become integrated into our social milieu, it is imperative to establish an understanding of whether and how humans perceive emotion in artificial agents. In this review, we incorporate recent findings from social robotics, virtual reality, psychology, and neuroscience to examine how people recognize and respond to emotions displayed by artificial agents. First, we review how people perceive emotions expressed by an artificial agent, such as facial and bodily expressions and vocal tone. Second, we evaluate the similarities and differences in the consequences of perceived emotions in artificial compared to human agents. Besides accurately recognizing the emotional state of an artificial agent, it is critical to understand how humans respond to those emotions. Does interacting with an angry robot induce the same responses in people as interacting with an angry person? Similarly, does watching a robot rejoice when it wins a game elicit similar feelings of elation in the human observer? Here we provide an overview of the current state of emotion expression and perception in social robotics, as well as a clear articulation of the challenges and guiding principles to be addressed as we move ever closer to truly emotional artificial agents

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the “like me” hypothesis, and discuss the key role played by the Theory‐of‐Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long‐term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents

Social cognition in the age of human–robot interaction

Artificial intelligence advances have led to robots endowed with increasingly sophisticated social abilities. These machines speak to our innate desire to perceive social cues in the environment, as well as the promise of robots enhancing our daily lives. However, a strong mismatch still exists between our expectations and the reality of social robots. We argue that careful delineation of the neurocognitive mechanisms supporting human–robot interaction will enable us to gather insights critical for optimising social encounters between humans and robots. To achieve this, the field must incorporate human neuroscience tools including mobile neuroimaging to explore long-term, embodied human–robot interaction in situ. New analytical neuroimaging approaches will enable characterisation of social cognition representations on a finer scale using sensitive and appropriate categorical comparisons (human, animal, tool, or object). The future of social robotics is undeniably exciting, and insights from human neuroscience research will bring us closer to interacting and collaborating with socially sophisticated robots

From social brains to social robots: applying neurocognitive insights to human-robot interaction

Amidst the fourth industrial revolution, social robots are resolutely moving from fiction to reality. With sophisticated artificial agents becoming ever more ubiquitous in daily life, researchers across different fields are grappling with the questions concerning how humans perceive and interact with these agents and the extent to which the human brain incorporates intelligent machines into our social milieu. This theme issue surveys and discusses the latest findings, current challenges and future directions in neuroscience- and psychology-inspired human–robot interaction (HRI). Critical questions are explored from a transdisciplinary perspective centred around four core topics in HRI: technical solutions for HRI, development and learning for HRI, robots as a tool to study social cognition, and moral and ethical implications of HRI. Integrating findings from diverse but complementary research fields, including social and cognitive neurosciences, psychology, artificial intelligence and robotics, the contributions showcase ways in which research from disciplines spanning biological sciences, social sciences and technology deepen our understanding of the potential and limits of robotic agents in human social life

The relation between bystanders' behavioral reactivity to distress and later helping behavior during a violent conflict in virtual reality

The occurrence of helping behavior is thought to be automatically triggered by reflexive reactions and promoted by intuitive decisions. Here, we studied whether reflexive reactions to an emergency situation are associated with later helping behavior in a different situation, a violent conflict. First, 29 male supporters of F.C. Barcelona performed a cued-reaction time task with a low and high cognitive load manipulation, to tap into reflexive and reflective processes respectively, during the observation of an emergency. Next, participants entered a bar in Virtual Reality and had a conversation with a virtual fellow supporter. During this conversation, a virtual Real Madrid supporter entered and started an aggressive argument with the fellow supporter that escalated into a physical fight. Verbal and physical interventions of the participant served as measures of helping behavior. Results showed that faster responses to an emergency situation during low, but not during high cognitive load, were associated with more interventions during the violent conflict. However, a tendency to describe the decision to act during the violent conflict as intuitive and reflex-like was related to more interventions. Further analyses revealed that a disposition to experience sympathy, other-oriented feelings during distressful situations, was related to self-reported intuitive decision-making, a reduced distance to the perpetrator, and higher in the intervening participants. Taken together, these results shed new light on helping behavior and are consistent with the notion of a motivational system in which the act of helping is dependent on a complex interplay between intuitive, reflexive and deliberate, reflective processes

A neurocognitive investigation of the impact of socializing with a robot on empathy for pain

To what extent can humans form social relationships with robots? In the present study, we combined functional neuroimaging with a robot socializing intervention to probe the flexibility of empathy, a core component of social relationships, towards robots. Twenty-six individuals underwent identical fMRI sessions before and after being issued a social robot to take home and interact with over the course of a week. While undergoing fMRI, participants observed videos of a human actor or a robot experiencing pain or pleasure in response to electrical stimulation. Repetition suppression of activity in the pain network, a collection of brain regions associated with empathy and emotional responding, was measured to test whether socializing with a social robot leads to greater overlap in neural mechanisms when observing human and robotic agents experiencing pain or pleasure. In contrast to our hypothesis, functional region-of-interest analyses revealed no change in neural overlap for agents after the socializing intervention. Similarly, no increase in activation when observing a robot experiencing pain emerged post-socializing. Whole-brain analysis showed that, before the socializing intervention, superior parietal and early visual regions are sensitive to novel agents, while after socializing, medial temporal regions show agent sensitivity. A region of the inferior parietal lobule was sensitive to novel emotions, but only during the pre-socializing scan session. Together, these findings suggest that a short socialization intervention with a social robot does not lead to discernible differences in empathy towards the robot, as measured by behavioural or brain responses. We discuss the extent to which long-term socialization with robots might shape social cognitive processes and ultimately our relationships with these machines. This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’

Trait dominance promotes reflexive staring at masked angry body postures

It has been shown that dominant individuals sustain eye-contact when non-consciously confronted with angry faces, suggesting reflexive mechanisms underlying dominance behaviors. However, dominance and submission can be conveyed and provoked by means of not only facial but also bodily features. So far few studies have investigated the interplay of body postures with personality traits and behavior, despite the biological relevance and ecological validity of these postures. Here we investigate whether non-conscious exposure to bodily expressions of anger evokes reflex-like dominance behavior. In an interactive eye-tracking experiment thirty-two participants completed three social dominance tasks with angry, happy and neutral facial, bodily and face and body compound expressions that were masked from consciousness. We confirmed our predictions of slower gaze-aversion from both non-conscious bodily and compound expressions of anger compared to happiness in high dominant individuals. Results from a follow-up experiment suggest that the dominance behavior triggered by exposure to bodily anger occurs with basic detection of the category, but not recognition of the emotional content. Together these results suggest that dominant staring behavior is reflexively driven by non-conscious perception of the emotional content and triggered by not only facial but also bodily expression of anger

Exploring the relationship between anthropomorphism and theory-of-mind in brain and behaviour

The process of understanding the minds of other people, such as their emotions and intentions, is mimicked when individuals try to understand an artificial mind. The assumption is that anthropomorphism, attributing human-like characteristics to non-human agents and objects, is an analogue to theory-of-mind, the ability to infer mental states of other people. Here, we test to what extent these two constructs formally overlap. Specifically, using a multi-method approach, we test if and how anthropomorphism is related to theory-of-mind using brain (Experiment 1) and behavioural (Experiment 2) measures. In a first exploratory experiment, we examine the relationship between dispositional anthropomorphism and activity within the theory-of-mind brain network (n = 108). Results from a Bayesian regression analysis showed no consistent relationship between dispositional anthropomorphism and activity in regions of the theory-of-mind network. In a follow-up, pre-registered experiment, we explored the relationship between theory-of-mind and situational and dispositional anthropomorphism in more depth. Participants (n = 311) watched a short movie while simultaneously completing situational anthropomorphism and theory-of-mind ratings, as well as measures of dispositional anthropomorphism and general theory-of-mind. Only situational anthropomorphism predicted the ability to understand and predict the behaviour of the film's characters. No relationship between situational or dispositional anthropomorphism and general theory-of-mind was observed. Together, these results suggest that while the constructs of anthropomorphism and theory-of-mind might overlap in certain situations, they remain separate and possibly unrelated at the personality level. These findings point to a possible dissociation between brain and behavioural measures when considering the relationship between theory-of-mind and anthropomorphism