The perception of emotion in artificial agents

Given recent technological developments in robotics, artificial intelligence and virtual reality, it is perhaps unsurprising that the arrival of emotionally expressive and reactive artificial agents is imminent. However, if such agents are to become integrated into our social milieu, it is imperative to establish an understanding of whether and how humans perceive emotion in artificial agents. In this review, we incorporate recent findings from social robotics, virtual reality, psychology, and neuroscience to examine how people recognize and respond to emotions displayed by artificial agents. First, we review how people perceive emotions expressed by an artificial agent, such as facial and bodily expressions and vocal tone. Second, we evaluate the similarities and differences in the consequences of perceived emotions in artificial compared to human agents. Besides accurately recognizing the emotional state of an artificial agent, it is critical to understand how humans respond to those emotions. Does interacting with an angry robot induce the same responses in people as interacting with an angry person? Similarly, does watching a robot rejoice when it wins a game elicit similar feelings of elation in the human observer? Here we provide an overview of the current state of emotion expression and perception in social robotics, as well as a clear articulation of the challenges and guiding principles to be addressed as we move ever closer to truly emotional artificial agents

A neurocognitive investigation of the impact of socializing with a robot on empathy for pain

To what extent can humans form social relationships with robots? In the present study, we combined functional neuroimaging with a robot socializing intervention to probe the flexibility of empathy, a core component of social relationships, towards robots. Twenty-six individuals underwent identical fMRI sessions before and after being issued a social robot to take home and interact with over the course of a week. While undergoing fMRI, participants observed videos of a human actor or a robot experiencing pain or pleasure in response to electrical stimulation. Repetition suppression of activity in the pain network, a collection of brain regions associated with empathy and emotional responding, was measured to test whether socializing with a social robot leads to greater overlap in neural mechanisms when observing human and robotic agents experiencing pain or pleasure. In contrast to our hypothesis, functional region-of-interest analyses revealed no change in neural overlap for agents after the socializing intervention. Similarly, no increase in activation when observing a robot experiencing pain emerged post-socializing. Whole-brain analysis showed that, before the socializing intervention, superior parietal and early visual regions are sensitive to novel agents, while after socializing, medial temporal regions show agent sensitivity. A region of the inferior parietal lobule was sensitive to novel emotions, but only during the pre-socializing scan session. Together, these findings suggest that a short socialization intervention with a social robot does not lead to discernible differences in empathy towards the robot, as measured by behavioural or brain responses. We discuss the extent to which long-term socialization with robots might shape social cognitive processes and ultimately our relationships with these machines. This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’

Virtual humans and Photorealism: The effect of photorealism of interactive virtual humans in clinical virtual environment on affective responses

The ability of realistic vs stylized representations of virtual characters to elicit emotions in users has been an open question for researchers and artists alike. We designed and performed a between subjects experiment using a medical virtual reality simulation to study the differences in the emotions aroused in participants while interacting with realistic and stylized virtual characters. The experiment included three conditions each of which presented a different representation of the virtual character namely; photo-realistic, non-photorealistic cartoon-shaded and non-photorealistic charcoal-sketch. The simulation used for the experiment, called the Rapid Response Training System was developed to train nurses to identify symptoms of rapid deterioration in patients. The emotional impact of interacting with the simulation on the participants was measured via both subjective and objective metrics. Quantitative objective measures were gathered using skin Electrodermal Activity (EDA) sensors, and quantitative subjective measures included Differential Emotion Survey (DES IV), Positive and Negative Affect Schedule (PANAS), and the co-presence or social presence questionnaire. The emotional state of the participants was analyzed across four distinct time steps during which the medical condition of the virtual patient deteriorated, and was contrasted to a baseline affective state. The data from the EDA sensors indicated that the mean level of arousal was highest in the charcoal-sketch condition, lowest in the realistic condition, with responses in the cartoon-shaded condition was in the middle. Mean arousal responses also seemed to be consistent in both the cartoon-shaded and charcoal-sketch conditions across all time steps, while the mean arousal response of participants in the realistic condition showed a significant drop from time step 1 through time step 2, corresponding to the deterioration of the virtual patient. Mean scores of participants in the DES survey seems to suggest that participants in the realistic condition elicited a higher emotional response than participants in both non-realistic conditions. Within the non-realistic conditions, participants in the cartoon-shaded condition seemed to elicit a higher emotional response than those in the charcoal-sketch condition

Factors of Emotion and Affect in Designing Interactive Virtual Characters

The Arts: 1st Place (The Ohio State University Edward F. Hayes Graduate Research Forum)This paper represents a review of literature concerning factors of affective interactive virtual character design. Affect and it's related concepts are defined followed by a detail of work being conducted in relevant areas such as design, animation, robotics. The intent of this review as to inform the author on overlapping concepts in fields related to affective design in order to apply these concepts interactive character development.A three-year embargo was granted for this item

Making Humanitarian Crises

This open access collection of essays explores the emotional agency of images in the construction of ‘humanitarian crises’ from the nineteenth century to the present. Using the prism of the histories of emotions and the senses, the chapters examine the pivotal role images have in shaping cultural, social and political reactions to the suffering of others and to the establishment of the international networks of solidarity. Questioning certain emotions assumed to underlie humanitarianism such as sympathy, empathy and compassion, they demonstrate how the experience of such emotions has shifted over time. Understanding images as emotional objects, contributors from a wide horizon of disciplines explore how their production, circulation and reception has been crucial to the perception of humanitarian crises in a long-term historical perspective

Making Humanitarian Crises

This open access collection of essays explores the emotional agency of images in the construction of ‘humanitarian crises’ from the nineteenth century to the present. Using the prism of the histories of emotions and the senses, the chapters examine the pivotal role images have in shaping cultural, social and political reactions to the suffering of others and to the establishment of the international networks of solidarity. Questioning certain emotions assumed to underlie humanitarianism such as sympathy, empathy and compassion, they demonstrate how the experience of such emotions has shifted over time. Understanding images as emotional objects, contributors from a wide horizon of disciplines explore how their production, circulation and reception has been crucial to the perception of humanitarian crises in a long-term historical perspective

Assessing children’s empathy through a Spanish adaptation of the Basic Empathy Scale: parent’s and child’s report forms

The aim of the current research was to study cognitive and affective empathy in children aged 6–12 years old, and their associations with children’s family environment and social adjustment. For this purpose, we developed the Spanish version of the Basic Empathy Scale (BES), self- and parent-report forms. Factorial analyses confirmed a two-component model of empathy in both self- and parent-report forms. Concordance between parent–child measures of empathy was low for cognitive and affective factors. Analyses of variance on the cognitive and affective components brought a significant effect of age for self-reported cognitive empathy, with older children scoring higher than younger ones. Gender brought out a significant principal effect for self-reported affective empathy, with girls scoring higher than boys. No other main effects were found for age and gender for the rest of the factors analyzed. Children’s empathy was associated with socioeconomic status and other family socialization processes, as well as children’ social behaviors. Overall the new measures provided a coherent view of empathy in middle childhood and early adolescence when measured through self and parent reports, and illustrate the similarity of the validity of the BES in a European-Spanish culture

How emotions and social interaction affect our food experience

The initial intent of this research was to understand whether there were possible interferences between the perception – the sensorial response of the individual when approaching food – and the effective, objective quality of the food Our aim was to investigate whether, at the moment a person is about to experience food, some external factor might influence the individual’s subsequent choice of food. We also wondered whether external factors might render a dish more or less appetizing. Initially we thought about investigating "external factors" such as changes of (exclusively) visual sensory aspects that would directly influence the appeal of food, such as color or size. Later, and during the course of our research, we decided to change our approach and investigate whether there might also be factors unrelated to the food that influence the perception of taste and the subsequent desirability of the food. For a given food, just before its consumption, we asked ourselves which environmental, external human factors might affect the internal subjective representation sufficiently to change the entire food experience. Our approach did not take into consideration a human environment where there was consumption of food, which has already been extensively investigated in numerous studies, but only the interference of the mere sight of an emotion relayed by others’ facial expressions. This research hypothesis is based on the concept of heterosis. We are interested in the method by which to approach a path of heterosis of scientific thought. Starting from the genetic concept of "hybrid vigor", we asked ourselves about the power of a multidisciplinary approach among different scientific disciplines, especially between food science and technology (presence of food), neuroscience (recent discoveries in the field of mirror neurons) and behavioral psychology (changes in food choice). In the field of genetics, heterosis refers to crossbreeding between unrelated individuals. The term heterosis is synonymous with interspecific hybridization, and is the opposite of inbreeding: in the scientific field, according to our approach and idea for the thesis, inbreeding signifies continuing to investigate within the same field of study, without any aperture towards other disciplines, or at least without assessing whether there might be possible interactions of an interdisciplinary nature. The population (gene pool) deriving from heterosis is a genotype which increases the frequency of heterozygosity, which means an increase in the number of loci with different alleles for the gene for the same character. This involves the generation of advantages that improve the fertility and genetics of the species. On the contrary, inbreeding increases the homozygosity, i.e. the presence of identical alleles at the same locus, and this is to the detriment of future genetic improvement. Heterosis is associated with the observed phenomenon known as hybrid vigor, in which the individual is the product of the coupling characteristics of a particularly vigorous phenotype: for example, there is an increase in stature, enhanced fertility, and a stronger resistance to disease. On the other hand, in the case of inbreeding one finds inbreeding depression, in which, among other things, there is an increase in the frequency of genetic diseases and a reduction of vigor and stature. The parallelism that we assumed in the scientific disciplines sees, in the field of interdisciplinary studies, a multiplier effect of discoveries and insights to the benefit of specific individual fields of research. The hybridization of different fields of science can generate new and unexplored fields of research, which may (this is our hypothesis) lead to new disciplines that are the result of the hybridization of such fields. For example: intelligent eating might be a wide new field of research involving neuroscience, psychology, medicine and food science and technology. The underlying theme of this thesis project, supported by comprehensive bibliographic database, is clearly of a hybrid nature, also in the formulation and planning of behavioral experiments. We followed the exploration of such hybridization of science fields without ever going into the specifics of one or the other discipline, but trying to maintain its cross-cutting nature. It is our hope that new fields of research, and even new branches of studies, might result from the heterosis of scientific disciplines, and this effort is intended to be a modest, experimental start whose ambition is to inspire a future in which every field of science has an internal development, specific and specialized, and one or more parallel multidisciplinary developments, each with a precise logic for the development and evolution of the holistic understanding of man.openDottorato di ricerca in Scienze degli alimentiopenRizzato, Matte

Haunting Images: Differential Perception and Emotional Response to the Archetypes of News Photography: A Study of Visual Reception Factored by Gender and Expertise

This dissertation explores how and why certain news photographs become memorable. Although researchers believe news photos count as forms of media expression, no one knows how influential these images really are in shaping societal attitudes. Social constructionist critics have argued that iconic images are pervasive markers of American collective memory. While icons have become the subject of intense media study, critics have ignored the presence of image archetypes that fall outside of the boundaries of the American iconic canon. They have also followed a top-down procedure of interpretation rather than a bottom-up method of collecting data from actual subjects. As I define it, the news image archetype is an authentically captured image of a human predicament of the greatest magnitude and seriousness showing conflict, tragedy, and occasionally, triumph. Visually these images communicate through physical gestures and facial expressions either directly, when faces are visible, or by implication in panoramic shots. Archetypal images can be iconic but need not be. Whereas icons are presumed to appeal to "everybody" by modeling ideology and "civic performance," archetypes need not exhibit any particular ideology. The common thread is more universally human than political. For this reason their appeal tends to be trans-cultural. This mixed-method study tests audience response to 41 outstanding news photographs including iconic, archetypal and ordinary examples. The purpose is to ascertain whether archetypal images can be distinguished and recalled as outstanding exemplars outside the iconic category; whether image quality preferences vary by visual expertise and gender; and how study subjects "read" the archetype. Using 2X2 ANOVA design, I studied four independent groups: male/female, visual expert/visual non-expert; n = 113. Study data indicate a convergence of ranking preference for some non-iconic archetypes that were rated as highly as famous icons. However, the strongest results show a convergence as to which image qualities (e.g., aesthetics, newsworthiness, emotional arousal etc.) were most important to viewers. The study found statistically significant differences of judgment on image qualities factored by gender and expertise. Qualitative results provided rich insights on factors affecting viewer response while composite data suggest multiple lines of future research

Stretching the Vitruvian Man: Investigating Affective and Representational Arts-based Methodologies Towards Theorizing a More Humanistic Model of Medicine

Westernized medicine can be said to illustrate its history and structure, as well as its current understanding of the capacity and appearance of the human through its visual representations of the body. Scientific images, this paper argues, become a site for interrogating the tangle of idealism, truth, objectivity and knowledge in how knowledge is actively used, replicated, paralleled and otherwise functions. First, asking how depictions of the medicalized body inform the epistemological foundations of medicine, and to what end, this work opens up the question of methodology, arguing that the integration of the modes of arts-based practices can bring medicine toward a much more realistic picture of the world. A parallel argument is a similarly concentrated interrogation of the affective quality of arts-based methodology, which is commonly understood to be the nucleus of work on the political dimensions of non-representational theory. I complicate the dominant scholarly preference for an ontologically rooted affect theory, finding it theoretically non-viable for art and humanistic medicine by thinking through subjectivity, autobiographical accounts of illness and epistemological flexibility. I see a path forward using a biologically and evolutionarily rooted affect theory, noting the ethical implications of its differences for a humanistic approach to medicine