Bringing Human Robot Interaction towards _Trust and Social Engineering

Robots started their journey in books and movies; nowadays, they are becoming an important part of our daily lives: from industrial robots, passing through entertainment robots, and reaching social robotics in fields like healthcare or education. An important aspect of social robotics is the human counterpart, therefore, there is an interaction between the humans and robots. Interactions among humans are often taken for granted as, since children, we learn how to interact with each other. In robotics, this interaction is still very immature, however, critical for a successful incorporation of robots in society. Human robot interaction (HRI) is the domain that works on improving these interactions. HRI encloses many aspects, and a significant one is trust. Trust is the assumption that somebody or something is good and reliable; and it is critical for a developed society. Therefore, in a society where robots can part, the trust they could generate will be essential for cohabitation. A downside of trust is overtrusting an entity; in other words, an insufficient alignment of the projected trust and the expectations of a morally correct behaviour. This effect could negatively influence and damage the interactions between agents. In the case of humans, it is usually exploited by scammers, conmen or social engineers - who take advantage of the people's overtrust in order to manipulate them into performing actions that may not be beneficial for the victims. This thesis tries to shed light on the development of trust towards robots, how this trust could become overtrust and be exploited by social engineering techniques. More precisely, the following experiments have been carried out: (i) Treasure Hunt, in which the robot followed a social engineering framework where it gathered personal information from the participants, improved the trust and rapport with them, and at the end, it exploited that trust manipulating participants into performing a risky action. (ii) Wicked Professor, in which a very human-like robot tried to enforce its authority to make participants obey socially inappropriate requests. Most of the participants realized that the requests were morally wrong, but eventually, they succumbed to the robot'sauthority while holding the robot as morally responsible. (iii) Detective iCub, in which it was evaluated whether the robot could be endowed with the ability to detect when the human partner was lying. Deception detection is an essential skill for social engineers and professionals in the domain of education, healthcare and security. The robot achieved 75% of accuracy in the lie detection. There were also found slight differences in the behaviour exhibited by the participants when interacting with a human or a robot interrogator. Lastly, this thesis approaches the topic of privacy - a fundamental human value. With the integration of robotics and technology in our society, privacy will be affected in ways we are not used. Robots have sensors able to record and gather all kind of data, and it is possible that this information is transmitted via internet without the knowledge of the user. This is an important aspect to consider since a violation in privacy can heavily impact the trust. Summarizing, this thesis shows that robots are able to establish and improve trust during an interaction, to take advantage of overtrust and to misuse it by applying different types of social engineering techniques, such as manipulation and authority. Moreover, robots can be enabled to pick up different human cues to detect deception, which can help both, social engineers and professionals in the human sector. Nevertheless, it is of the utmost importance to make roboticists, programmers, entrepreneurs, lawyers, psychologists, and other sectors involved, aware that social robots can be highly beneficial for humans, but they could also be exploited for malicious purposes

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

More Than Machines? The Attribution of (In)Animacy to Robot Technology

We know that robots are just machines. Why then do we often talk about them as if they were alive? The author explores this fascinating phenomenon, providing a rich insight into practices of animacy (and inanimacy) attribution to robot technology: from science-fiction to robotics R&D, from science communication to media discourse, and from the theoretical perspectives of STS to the cognitive sciences. Taking an interdisciplinary perspective, and backed by a wealth of empirical material, the author shows how scientists, engineers, journalists - and everyone else - can face the challenge of robot technology appearing "a little bit alive" with a reflexive and yet pragmatic stance

More Than Machines?

We know that robots are just machines. Why then do we often talk about them as if they were alive? Laura Voss explores this fascinating phenomenon, providing a rich insight into practices of animacy (and inanimacy) attribution to robot technology: from science-fiction to robotics R&D, from science communication to media discourse, and from the theoretical perspectives of STS to the cognitive sciences. Taking an interdisciplinary perspective, and backed by a wealth of empirical material, Voss shows how scientists, engineers, journalists - and everyone else - can face the challenge of robot technology appearing »a little bit alive« with a reflexive and yet pragmatic stance

More Than Machines?

We know that robots are just machines. Why then do we often talk about them as if they were alive? Laura Voss explores this fascinating phenomenon, providing a rich insight into practices of animacy (and inanimacy) attribution to robot technology: from science-fiction to robotics R&D, from science communication to media discourse, and from the theoretical perspectives of STS to the cognitive sciences. Taking an interdisciplinary perspective, and backed by a wealth of empirical material, Voss shows how scientists, engineers, journalists - and everyone else - can face the challenge of robot technology appearing »a little bit alive« with a reflexive and yet pragmatic stance

Human—Robot companionship: A mixed-methods investigation

In recent years, the arts have brought robots to life in spectacular fashion. In popular fiction we have been presented with machines that can run, leap, fight, and (perhaps most impressively of all) robots which can ascend stairs with absolutely no trouble at all. Amidst these chaotic and often dystopian scenes, we are exposed to moments of humour and lightness – robots can be seen engaging in conversation, cracking jokes, and comforting someone in their time of need. In these relatively mundane moments (as we smile, laugh, and cry) the impression emerges that the robot is something special to the person depicted. Rather than simply being a household appliance, it appears to be something more: a sort of… friend. Returning from the pages and screens of fiction to the real world, we find human society ever more fractured, and the loneliness epidemic at large. Unsurprisingly, given the engaging depictions in popular fiction, the idea of robots for companionship and social support is gaining traction and garnering increasing research attention. In care homes, robot animals can be found cooing and purring in the laps of individuals with dementia, while in schools, friendly humanoid robots may be seen teaching social skills to children with additional needs. What remains unknown, though, is the extent to which people will grow fond of such ‘social robots’ over time, and if so, whether their relationships with these machines might ever resemble (or indeed, replace) those with other humans. Is a ‘robot friend’ the stuff of science-fiction, or could it someday soon become sciencereality? In this thesis, this question is explored from a range of perspectives using a variety of methods spanning lab-based experiments, online surveys, and focus groups. This thesis begins with an introduction to social robots, and an exploration of the background regarding the nature and importance of human social relationships. After introducing relevant theories, I highlight gaps in our understanding of human—robot companionship that I seek to explore through this thesis (Chapter 1). In the subsequent chapters, I present four empirical pieces of work, each offering a unique perspective on the subject. Specifically, in Chapter 2, I report results from a lab-based experiment in which a robot’s lights (located within its shoulders) were programmed to illuminate in a synchronous or asynchronous manner relative to a participant’s heart rate. I aimed to determine whether such a synchrony manipulation might increase prosocial behaviours and improve attitudes towards a social robot - based on prior work showing that experimentally-induced movement synchrony can improve rapport between people, and increase their liking of social robots (Hove & Risen, 2009; Lehmann et al., 2015, Mogan, Fischer & Bulbulia, 2017). Despite demonstrating no positive effect of the light manipulation, this study raises important questions regarding the complexities of defining and measuring attachment to a robot. In Chapter 3, I delve deeper into the qualitative data collected in Chapter 2 to build a more complete appreciation of the value of open questions – particularly in terms of method validation and understanding participants’ internal experiences. After this chapter, I shift perspective from a focus on humanoid robots (and manipulations based on human social behaviours), to human relationships with non-human companion animals. This shift was motivated by my desire to explore how non-human agents form deep and enduring social bonds with humans – as opposed to basing the thesis on human interpersonal relationships alone. Due to the success of dogs as companions, I conducted a study in which dog owners were asked to identify behaviours that they perceived as important to the bond with their dog (Chapter 4). Seven key themes emerged from this research, indicating the importance of attunement, communication, consistency and predictability, physical affection, positivity and enthusiasm, proximity, and shared activities. In the following chapter, I implement a selection of ‘desirable’ dog behaviours within an animal-inspired robot (Chapter 5). By showing the behaviours to members of the general public, and conducting focus groups, I gained deeper insights into the polarising nature of robot animals – not only in terms of how their behaviours are perceived, but also in terms of the roles people think robots should (and should not) hold. In addition to these themes, this final empirical chapter discusses insights regarding the high expectations people place upon robots, as well as public concerns around overdependence on robots, and privacy. By releasing these chapters to the HRI community (through publications or preprints) we sparked conversations within the HRI community – not only about the ethics of robot abuse studies, but also the potential value of qualitative approaches within the field. Our team was commended for publishing qualitative research, in a field heavily dominated by quantitative methods, and we have since been working to continue the conversations around the value of qualitative approaches. Specifically, we hosted the “Enriching HRI Research with Qualitative Methods” workshop at the International Journal on Social Robotics (2020) and launched a “Qualitative Research in HRI/HCI Discussion Group” online - allowing HRI researchers to discuss their work, and share relevant resources (e.g., events and publications). This thesis concludes by detailing work to be done moving forwards, to enhance our understanding of human—robot social relationships, and a broader discussion of our possible future with social robots (Chapter 6). Pulling from various disciplines (including psychology, cognitive science, human—robot interaction (HRI) Studies, robot ethics, and philosophy), this section concludes with consideration of potential consequences of companion technologies – not only for the individual, but perhaps for society as a whole, as we continue to grapple with questions concerning how much of science fiction we wish to welcome into our daily lives

Emotional body language synthesis for humanoid robots

Some of the chapters of this thesis are based on research published by the author. Chapter 4 is based on Marmpena M., Lim, A., and Dahl, T. S. (2018). How does the robot feel? Perception of valence and arousal in emotional body language. Paladyn, Journal of Behavioral Robotics, 9(1), 168-182. DOI: https://doi.org/10.1515/pjbr-2018-0012. Chapter 6 is based on Marmpena M., Lim, A., Dahl, T. S., and Hemion, N. (2019). Generating robotic emotional body language with Variational Autoencoders. In Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 545–551. DOI:10.1109/ACII.2019.8925459. Chapter 7 extends Marmpena M., Garcia, F., and Lim, A. (2020). Generating robotic emotional body language of targeted valence and arousal with Conditional Variational Autoencoders. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’20, page 357–359. DOI: https://doi.org/10.1145/3371382.3378360. The designed or generated robotic emotional body language expressions data presented in this thesis are publicly available: https://github.com/minamar/rebl-pepper-dataIn the next decade, societies will witness a rise in service robots deployed in social environments, such as schools, homes, or shops, where they will operate as assistants, public relation agents, or companions. People are expected to willingly engage and collaborate with these robots to accomplish positive outcomes. To facilitate collaboration, robots need to comply with the behavioural and social norms used by humans in their daily interactions. One such behavioural norm is the expression of emotion through body language. Previous work on emotional body language synthesis for humanoid robots has been mainly focused on hand-coded design methods, often employing features extracted from human body language. However, the hand-coded design is cumbersome and results in a limited number of expressions with low variability. This limitation can be at the expense of user engagement since the robotic behaviours will appear repetitive and predictable, especially in long-term interaction. Furthermore, design approaches strictly based on human emotional body language might not transfer effectively on robots because of their simpler morphology. Finally, most previous work is using six or fewer basic emotion categories in the design and the evaluation phase of emotional expressions. This approach might result in lossy compression of the granularity in emotion expression. The current thesis presents a methodology for developing a complete framework of emotional body language generation for a humanoid robot, intending to address these three limitations. Our starting point is a small set of animations designed by professional animators with the robot morphology in mind. We conducted an initial user study to acquire reliable dimensional labels of valence and arousal for each animation. In the next step, we used the motion sequences from these animations to train a Variational Autoencoder, a deep learning model, to generate numerous new animations in an unsupervised setting. Finally, we extended the model to condition the generative process with valence and arousal attributes, and we conducted a user study to evaluate the interpretability of the animations in terms of valence, arousal, and dominance. The results indicate moderate to strong interpretability

A Retro-Projected Robotic Head for Social Human-Robot Interaction

As people respond strongly to faces and facial features, both con- sciously and subconsciously, faces are an essential aspect of social robots. Robotic faces and heads until recently belonged to one of the following categories: virtual, mechatronic or animatronic. As an orig- inal contribution to the field of human-robot interaction, I present the R-PAF technology (Retro-Projected Animated Faces): a novel robotic head displaying a real-time, computer-rendered face, retro-projected from within the head volume onto a mask, as well as its driving soft- ware designed with openness and portability to other hybrid robotic platforms in mind. The work constitutes the first implementation of a non-planar mask suitable for social human-robot interaction, comprising key elements of social interaction such as precise gaze direction control, facial ex- pressions and blushing, and the first demonstration of an interactive video-animated facial mask mounted on a 5-axis robotic arm. The LightHead robot, a R-PAF demonstrator and experimental platform, has demonstrated robustness both in extended controlled and uncon- trolled settings. The iterative hardware and facial design, details of the three-layered software architecture and tools, the implementation of life-like facial behaviours, as well as improvements in social-emotional robotic communication are reported. Furthermore, a series of evalua- tions present the first study on human performance in reading robotic gaze and another first on user’s ethnic preference towards a robot face

Care robots in residential homes for elderly people: an ethical examination of deception, care, and consent

We are facing a dire social problem: although life expectancy is increasing, time spent living independently is not, meaning that the eldercare sector is experiencing a worrying shortfall of nursing staff - a problem which is only getting worse. Robots designed for caring purposes – carebots – present a possible solution: they can perform some of the work which has been hitherto undertaken by human nurses. But their introduction is not without problems. This thesis examines some pertinent questions relating to the introduction of carebots into residential homes for elderly people. Chapter 1 examines what robots are, and provides a way in which we can differentiate between robots of different types, helping us to understand what ethical issues are at stake for different types of robot. Chapter 2 focuses on what deception consists of, and discusses why deception and lying are often seen as impermissible. Chapter 3 discusses different types of robo-deception, and analyses both the likelihood and the normative significance of their occurring. Chapter 4 is a study of a particular form of robo-deception, which I call fake compassion. This is when robots appear to care for patients when in fact they do not: I examine the extent to which this is morally problematic. Chapter 5 examines dignity: what it is, and why it is important. Chapter 6 focuses on consent: its importance in different spheres, and how consent-seeking can promote autonomy, bodily integrity, dignity, and trust. Chapter 7 builds on the previous two chapters, and demonstrates that it is ethically essential that carebots (and human nurses) obtain patients' consent prior to providing care, because failing to do so can reduce their dignity, and these reductions can be cumulative and devastating. This thesis is not merely an interesting thought experiment or a work of science fiction; rather, it is a real-world necessity that carebots take appropriate actions which promote the dignity and best interests of patients: our grandparents, parents, and in time, us and our descendants