How Children with Autism Spectrum Disorder Recognize Facial Expressions Displayed by a Rear-Projection Humanoid Robot

Background: Children with Autism Spectrum Disorder (ASD) experience reduced ability to perceive crucial nonverbal communication cues such as eye gaze, gestures, and facial expressions. Recent studies suggest that social robots can be used as effective tools to improve communication and social skills in children with ASD. One explanation has been put forward by several studies that children with ASD feel more contented and motivated in systemized and predictable environment, like interacting with robots. Objectives: There have been few research studies evaluating how children with ASD perceive facial expression in humanoid robots but no research evaluating facial expression perception on a rear-projected (aka animation-based) facially-expressive humanoid robot, which provide more life-like expressions. This study evaluates how children with high functioning autism (HFA) differ from their typically developing (TD) peers in recognition of facial expressions demonstrated by a life-like rear-projected humanoid robot, which is more adjustable and flexible in terms of displaying facial expressions for further studies. Methods: Seven HFA and seven TD children and adolescents aged 7-16 participated in this study. The study uses Ryan, a rear-projection, life-like humanoid robot. Six basic emotional facial expressions (happy, sad, angry, disgust, surprised and fear) with four different intensities (25%, 50%, 75% and 100% in ascending order) were shown on Ryan’s face. Participants were asked to choose the expression they perceived among seven options (six basic emotions and none). Responses were recorded by a research assistant. Results were analyzed to obtain the accuracy of facial expression recognition in ASD and TD children on humanoid robot face. Results: We evaluated the intensity of expression in which participants required to reach the peak accuracy. They were best for happy and angry expressions in which the peak accuracy of 100% was reached with at least 50% of expression intensity. The same peak accuracy was reached for surprised and sad expressions in the intensity of 75% and 100%, respectively. But fear and disgust recognition accuracy never reached above 75%, even in the maximum intensity. The experiment is still in progress for TD children. Results will be compared to a TD sample and implication for intervention and clinical work will be discussed. Conclusions: Overall, these results show that children with ASD recognize negative expressions such as fear and disgust with a slightly lower accuracy than other expressions. On the other hand, during the test, children showed engagement and excitement toward the robot. Besides, most of the expressions were sufficiently recognizable for children in higher intensities, which means, Ryan, a rear projected life-like robot could be able to successfully communicate with children in terms of facial expression, though more investigations and improvements should be done. These results serve as a basis to advance the promising field of socially assistive robotics for autism therapy

The perception of emotion in artificial agents

Given recent technological developments in robotics, artificial intelligence and virtual reality, it is perhaps unsurprising that the arrival of emotionally expressive and reactive artificial agents is imminent. However, if such agents are to become integrated into our social milieu, it is imperative to establish an understanding of whether and how humans perceive emotion in artificial agents. In this review, we incorporate recent findings from social robotics, virtual reality, psychology, and neuroscience to examine how people recognize and respond to emotions displayed by artificial agents. First, we review how people perceive emotions expressed by an artificial agent, such as facial and bodily expressions and vocal tone. Second, we evaluate the similarities and differences in the consequences of perceived emotions in artificial compared to human agents. Besides accurately recognizing the emotional state of an artificial agent, it is critical to understand how humans respond to those emotions. Does interacting with an angry robot induce the same responses in people as interacting with an angry person? Similarly, does watching a robot rejoice when it wins a game elicit similar feelings of elation in the human observer? Here we provide an overview of the current state of emotion expression and perception in social robotics, as well as a clear articulation of the challenges and guiding principles to be addressed as we move ever closer to truly emotional artificial agents

How Expressiveness of a Robotic Tutor is Perceived by Children in a Learning Environment

We present a study investigating the expressiveness of two different types of robots in a tutoring task. The robots used were i) the EMYS robot, with facial expression capabilities, and ii) the NAO robot, without facial expressions but able to perform expressive gestures. Preliminary results show that the NAO robot was perceived to be more friendly, pleasant and empathic than the EMYS robot as a tutor in a learning environment

Towards Inferring Users' Impressions of Robot Performance in Navigation Scenarios

Human impressions of robot performance are often measured through surveys. As a more scalable and cost-effective alternative, we study the possibility of predicting people's impressions of robot behavior using non-verbal behavioral cues and machine learning techniques. To this end, we first contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in a Virtual Reality simulation, together with impressions of robot performance provided by users on a 5-point scale. Second, we contribute analyses of how well humans and supervised learning techniques can predict perceived robot performance based on different combinations of observation types (e.g., facial, spatial, and map features). Our results show that facial expressions alone provide useful information about human impressions of robot performance; but in the navigation scenarios we tested, spatial features are the most critical piece of information for this inference task. Also, when evaluating results as binary classification (rather than multiclass classification), the F1-Score of human predictions and machine learning models more than doubles, showing that both are better at telling the directionality of robot performance than predicting exact performance ratings. Based on our findings, we provide guidelines for implementing these predictions models in real-world navigation scenarios