A Comparison of Avatar-, Video-, and Robot-Mediated Interaction on Users’ Trust in Expertise

Communication technologies are becoming increasingly diverse in form and functionality. A central concern is the ability to detect whether others are trustworthy. Judgments of trustworthiness rely, in part, on assessments of non-verbal cues, which are affected by media representations. In this research, we compared trust formation on three media representations. We presented 24 participants with advisors represented by two of the three alternate formats: video, avatar, or robot. Unknown to the participants, one was an expert, and the other was a non-expert. We observed participants’ advice-seeking behavior under risk as an indicator of their trust in the advisor. We found that most participants preferred seeking advice from the expert, but we also found a tendency for seeking robot or video advice. Avatar advice, in contrast, was more rarely sought. Users’ self-reports support these findings. These results suggest that when users make trust assessments, the physical presence of the robot representation might compensate for the lack of identity cues

Comparing video, avatar, and robot mediated communication: Pros and cons of embodiment

Abstract. In recent years, studies have begun on robot conferencing as a new telecommunication medium. In robot conferencing, people talk with a remote conversation partner through teleoperated robots which present the bodily motions of the partner with a physical embodiment. However, the effects of physical embodiment on distant communication had not yet been demonstrated. In this study, to find the effects, we conducted an experiment in which subjects talked with a partner through robots and various existing communication media (e.g. voice, avatar and video chats). As a result, we found that the physical embodiment enhanced social telepresence, i.e., the sense of resembling face-toface interaction. Furthermore, the result implied that physical embodiment built the sense of tension as in the case of a first face-to-face meeting

A Survey of Multi-Agent Human-Robot Interaction Systems

This article presents a survey of literature in the area of Human-Robot Interaction (HRI), specifically on systems containing more than two agents (i.e., having multiple humans and/or multiple robots). We identify three core aspects of ``Multi-agent" HRI systems that are useful for understanding how these systems differ from dyadic systems and from one another. These are the Team structure, Interaction style among agents, and the system's Computational characteristics. Under these core aspects, we present five attributes of HRI systems, namely Team size, Team composition, Interaction model, Communication modalities, and Robot control. These attributes are used to characterize and distinguish one system from another. We populate resulting categories with examples from recent literature along with a brief discussion of their applications and analyze how these attributes differ from the case of dyadic human-robot systems. We summarize key observations from the current literature, and identify challenges and promising areas for future research in this domain. In order to realize the vision of robots being part of the society and interacting seamlessly with humans, there is a need to expand research on multi-human -- multi-robot systems. Not only do these systems require coordination among several agents, they also involve multi-agent and indirect interactions which are absent from dyadic HRI systems. Adding multiple agents in HRI systems requires advanced interaction schemes, behavior understanding and control methods to allow natural interactions among humans and robots. In addition, research on human behavioral understanding in mixed human-robot teams also requires more attention. This will help formulate and implement effective robot control policies in HRI systems with large numbers of heterogeneous robots and humans; a team composition reflecting many real-world scenarios.Comment: 23 pages, 7 figure

Studying Eye Gaze of Children with Autism Spectrum Disorders in Interaction with a Social Robot

Children with Autism Spectrum Disorders (ASDs) experience deficits in verbal and nonverbal communication skills including motor control, emotional facial expressions, and eye gaze attention. In this thesis, we focus on studying the feasibility and effectiveness of using a social robot, called NAO, at modeling and improving the social responses and behaviors of children with autism. In our investigation, we designed and developed two protocols to fulfill this mission. Since eye contact and gaze responses are important non-verbal cues in human\u27s social communication and as the majority of individuals with ASD have difficulties regulating their gaze responses, in this thesis we have mostly focused on this area. In Protocol 1 (eye gaze duration and shifting frequency are analyzed in this protocol), we designed two social games (i.e. NAO Spy and Find the Suspect) and recruited 21 subjects (i.e. 14 ASD and seven Typically Developing (TD) children) ages between 7-17 years old to interact with NAO. All sessions were recorded using cameras and the videos were used for analysis. In particular, we manually annotated the eye gaze direction of children (i.e. gaze averted `0\u27 or gaze at robot `1\u27) in every frame of the videos within two social contexts (i.e. child speaking and child listening). Gaze fixation and gaze shifting frequency are analyzed, where both patterns are significantly improved or changed (more than half of the participants increased the eye contact duration time and decrease the eye gaze shifting during both games). The results confirms that the TD group has more gaze fixation as they are listening (71%) than while they are speaking (37%). However there is no significant difference between the average gaze fixations of ASD group. Besides using the statistical measures (i.e. gaze fixation and shifting), we statistically modeled the gaze responses of both groups (TD and ASD) using Markov models (e.g. Hidden Markov Model (HMM) and Variable-order Markov Model (VMM)). Using Markov based modeling allows us to analyze the sequence of gaze direction of ASD and TD groups for two social conversational sessions (Child Speaking and Listening). The results of our experiments show that for the `Child Speaking\u27 segments, HMM can distinguish and recognize the differences of gaze patterns of TD and ASD groups accurately (79%). In addition, to evaluate the effect of history of eye gaze in the gaze responses, the VMM technique was employed to model the effects of different length of sequential data. The results of VMM demonstrate that, in general, the first order system (VMM with order D=1) can reliably represent the differences between the gaze patterns of TD and ASD group. Besides that, the experimental results confirm that VMM is more reliable and accurate for modeling the gaze responses of Child Listening sessions than the Child Speaking one. Protocol 2 contains five sub-sessions targeted intervention of different social skills: verbal communication, joint attention, eye gaze attention, facial expressions recognition/imitation. The objective of this protocol is to provide intervention sessions based on the needs of children diagnosed with ASD. Therefore each participant attended in three times of baseline sessions for evaluate his/her existing social skill and behavioral response, when the study began. In this protocol the behavioral responses of every child is recorded in each intervention session where feedbacks are focused on improving their social skills if they lack one. For example if they are not good at recognizing facial expression, we give them feedback on how every facial expression looks like and ask them to recognize them correctly while we do not feedback on other social skills. Our experimental results show that customizing the human-robot interaction would improve the social skills of children with ASD

Shared Perception in Human-Robot Interaction

Interaction can be seen as a composition of perspectives: the integration of perceptions, intentions, and actions on the environment two or more agents share. For an interaction to be effective, each agent must be prone to “sharedness”: being situated in a common environment, able to read what others express about their perspective, and ready to adjust one’s own perspective accordingly. In this sense, effective interaction is supported by perceiving the environment jointly with others, a capability that in this research is called Shared Perception. Nonetheless, perception is a complex process that brings the observer receiving sensory inputs from the external world and interpreting them based on its own, previous experiences, predictions, and intentions. In addition, social interaction itself contributes to shaping what is perceived: others’ attention, perspective, actions, and internal states may also be incorporated into perception. Thus, Shared perception reflects the observer's ability to integrate these three sources of information: the environment, the self, and other agents. If Shared Perception is essential among humans, it is equally crucial for interaction with robots, which need social and cognitive abilities to interact with humans naturally and successfully. This research deals with Shared Perception within the context of Social Human-Robot Interaction (HRI) and involves an interdisciplinary approach. The two general axes of the thesis are the investigation of human perception while interacting with robots and the modeling of robot’s perception while interacting with humans. Such two directions are outlined through three specific Research Objectives, whose achievements represent the contribution of this work. i) The formulation of a theoretical framework of Shared Perception in HRI valid for interpreting and developing different socio-perceptual mechanisms and abilities. ii) The investigation of Shared Perception in humans focusing on the perceptual mechanism of Context Dependency, and therefore exploring how social interaction affects the use of previous experience in human spatial perception. iii) The implementation of a deep-learning model for Addressee Estimation to foster robots’ socio-perceptual skills through the awareness of others’ behavior, as suggested in the Shared Perception framework. To achieve the first Research Objective, several human socio-perceptual mechanisms are presented and interpreted in a unified account. This exposition parallels mechanisms elicited by interaction with humans and humanoid robots and aims to build a framework valid to investigate human perception in the context of HRI. Based on the thought of D. Davidson and conceived as the integration of information coming from the environment, the self, and other agents, the idea of "triangulation" expresses the critical dynamics of Shared Perception. Also, it is proposed as the functional structure to support the implementation of socio-perceptual skills in robots. This general framework serves as a reference to fulfill the other two Research Objectives, which explore specific aspects of Shared Perception. For what concerns the second Research Objective, the human perceptual mechanism of Context Dependency is investigated, for the first time, within social interaction. Human perception is based on unconscious inference, where sensory inputs integrate with prior information. This phenomenon helps in facing the uncertainty of the external world with predictions built upon previous experience. To investigate the effect of social interaction on such a mechanism, the iCub robot has been used as an experimental tool to create an interactive scenario with a controlled setting. A user study based on psychophysical methods, Bayesian modeling, and a neural network analysis of human results demonstrated that social interaction influenced Context Dependency so that when interacting with a social agent, humans rely less on their internal models and more on external stimuli. Such results are framed in Shared Perception and contribute to revealing the integration dynamics of the three sources of Shared Perception. The others’ presence and social behavior (other agents) affect the balance between sensory inputs (environment) and personal history (self) in favor of the information shared with others, that is, the environment. The third Research Objective consists of tackling the Addressee Estimation problem, i.e., understanding to whom a speaker is talking, to improve the iCub social behavior in multi-party interactions. Addressee Estimation can be considered a Shared Perception ability because it is achieved by using sensory information from the environment, internal representations of the agents’ position, and, more importantly, the understanding of others’ behavior. An architecture for Addressee Estimation is thus designed considering the integration process of Shared Perception (environment, self, other agents) and partially implemented with respect to the third element: the awareness of others’ behavior. To achieve this, a hybrid deep-learning (CNN+LSTM) model is developed to estimate the speaker-robot relative placement of the addressee based on the non-verbal behavior of the speaker. Addressee Estimation abilities based on Shared Perception dynamics are aimed at improving multi-party HRI. Making robots aware of other agents’ behavior towards the environment is the first crucial step for incorporating such information into the robot’s perception and modeling Shared Perception

WEHST: Wearable Engine for Human-Mediated Telepresence

This dissertation reports on the industrial design of a wearable computational device created to enable better emergency medical intervention for situations where electronic remote assistance is necessary. The design created for this doctoral project, which assists practices by paramedics with mandates for search-and-rescue (SAR) in hazardous environments, contributes to the field of human-mediated teleparamedicine (HMTPM). Ethnographic and industrial design aspects of this research considered the intricate relationships at play in search-and-rescue operations, which lead to the design of the system created for this project known as WEHST: Wearable Engine for Human-Mediated Telepresence. Three case studies of different teams were carried out, each focusing on making improvements to the practices of teams of paramedics and search-and-rescue technicians who use combinations of ambulance, airplane, and helicopter transport in specific chemical, biological, radioactive, nuclear and explosive (CBRNE) scenarios. The three paramedicine groups included are the Canadian Air Force 442 Rescue Squadron, Nelson Search and Rescue, and the British Columbia Ambulance Service Infant Transport Team. Data was gathered over a seven-year period through a variety of methods including observation, interviews, examination of documents, and industrial design. The data collected included physiological, social, technical, and ecological information about the rescuers. Actor-network theory guided the research design, data analysis, and design synthesis. All of this leads to the creation of the WEHST system. As identified, the WEHST design created in this dissertation project addresses the difficulty case-study participants found in using their radios in hazardous settings. As the research identified, a means of controlling these radios without depending on hands, voice, or speech would greatly improve communication, as would wearing sensors and other computing resources better linking operators, radios, and environments. WEHST responds to this need. WEHST is an instance of industrial design for a wearable “engine” for human-situated telepresence that includes eight interoperable families of wearable electronic modules and accompanying textiles. These make up a platform technology for modular, scalable and adaptable toolsets for field practice, pedagogy, or research. This document details the considerations that went into the creation of the WEHST design

Cognitive impacts of social virtual reality: disentangling the virtual mere presence and audience effect

Researchers have investigated the impacts of social co-presence on the individual’s performance for over a century, finding that performance changes in a social setting when contrasted to performing alone – termed the social facilitation effect (SFE). Driven by the demand for realistic remote interaction, social technologies are currently aspiring to elicit a meaningful state of virtual co-presence. However, the virtual-SFE literature is currently inconclusive, especially when contrasting the AI versus human-driven SFE-impact. This thesis argues that current virtual-SFE findings can be elucidated by investigating SFE through its mechanisms: the feeling of being observed (audience effect: AE) and the sense of co-presence with another person (mere presence effect: MPE). The three experiments tested whether AE and MPE impact participants cognitive performance differently, depending on whether the companion is human-minded or AI-driven, during either remote videoconference or lab-based immersive virtual interaction. AE was predicted to be susceptible to human-minded companion impact, the MPE to be susceptible to the visual co-presence of any humanoid companion. Videoconference-based experiment one and two demonstrated that videoconference MPE and AE were facilitatory: MPE driven by the participants self-visual presence, not companion-visual presence and AE driven by human-minded companion as predicted. The immersive in-lab experiment three found MPE and AE were inhibitory: humanoid companion presence drove the MPE, and AE was irrespective of companion mind property. Overall, the findings supported the predictions that MPE and AE can be aroused independently by changing participants beliefs about their social-companion and their observed virtual co-presence, explaining some trends in current virtual-SFE literature. However, future studies should be mindful of virtual platform affordances, participants self-presence, and real-world testing-environment when testing and interpreting results. The sufficient level of virtual co-immersion and self-visual presence is required for virtual-SFE. Hopefully this research will pave the way towards greater understanding of virtual cognition and development of wellbeing-focused virtual-platforms