生命維持にかかわる生理現象を介した人間 : ロボットのコミュニケーションと身体情動モデルの設計

関西大学In this dissertation, we focus on physiological phenomena of robots as the expressive modality of their inner states and discuss the effectiveness of a robot expressing physiological phenomena, which are indispensable for living. We designed a body-emotion model showing the relationship between a) emotion as the inner state of the robot and b) physiological phenomena as physical changes, and we discuss the communication between humans and robots through involuntary physiological expression based on the model. In recent years, various robots for use in mental health care and communication support in medical/nursing care have been developed. The purpose of these systems is to enable communication between a robot and patients by an active approach of the robot through sound and body movement. In contrast to conventional approaches, our research is based on involuntary emotional expression through physiological phenomena of the robot. Physiological phenomena including breathing, heartbeat, and body temperature are essential functions for life activities, and these are closely related to the inner state of humans because physiological phenomena are caused by the emotional reaction of the limbic system transmitted via the autonomic nervous system. In human-robot communication through physical contact, we consider that physiological phenomena are one of the most important nonverbal modalities of the inner state as involuntary expressions. First, we focused on the robots\u27 expression of physiological phenomena, proposed the body-emotion model (BEM), which concerns the relationship between the inner state of robots and their involuntary physical reactions. We proposed a stuffed-toy robot system: BREAR―which has a mechanical structure to express the breathing, heartbeat, temperature and bodily movement. The result of experiment showed that a heartbeat, breathing and body temperature can express the robot\u27s living state and that the breathing speed is highly related to the robot\u27s emotion of arousal. We reviewed the experimental results and emotional generation mechanisms and discussed the design of the robot based on BEM. Based on our verification results, we determined that the design of the BEM-which involves the perception of the external situation, the matching with the memory, the change of the autonomic nervous parameter and the representation of the physiological phenomena - that is based on the relationship between the autonomic nervous system and emotional arousal is effective. Second, we discussed indirect communication between humans and robots through physiological phenomena - which consist of the breathing, heartbeats and body temperature - that express robots\u27 emotions. We set a situation with joint attention from the robot and user on emotional content and evaluated whether both the user\u27s emotional response to the content and the user\u27s impression of relationship between the user and the robot were changed by the physiological expressions of the robot. The results suggest that the physiological expression of the robot makes the user\u27s own emotions in the experience more excited or suppressed and that the robot\u27s expression increases impressions of closeness and sensitivity. Last, we discussed the future perspective of human-robot communication by physiological phenomena. Regarding the representation of the robots\u27 sense of life, it is thought that the user\u27s recognition that the robot is alive improves not only the moral effect on the understanding of the finiteness of life but also the attachment to the robot in long-term communication. Regarding the emotional expression mechanism based on life, it is expected that the robot can display a complicated internal state close to that of humans by combining intentionally expressed emotions and involuntary emotional expressions. If a robot can express a combination of realistic voluntary expressions, such as facial expressions and body movements, in combination with real involuntary expressions by using the real intentions and lying, it can be said that the robot has a more complicated internal state than that of a pet. By using a robot expressing a living state through physiological phenomena, it can be expected that the effect of mental care will exceed that of animal therapy, and we expect to provide care and welfare support in place of human beings

The power of affective touch within social robotics

There have been many leaps and bounds within social robotics, especially within human-robot interaction and how to make it a more meaningful relationship. This is traditionally accomplished through communicating via vision and sound. It has been shown that humans naturally seek interaction through touch yet the implications on emotions is unknown both in human-human interaction and social human-robot interaction. This thesis unpacks the social robotics community and the research undertaken to show a significant gap in the use of touch as a form of communication. The meaning behind touch will be investigated and what implication it has on emotions. A simplistic prototype was developed focusing on texture and breathing. This was used to carry out experiments to find out which combination of texture and movement felt natural. This proved to be a combination of synthetic fur and 14 breaths per minute. For human’s touch is said to be the most natural way of communicating emotions, this is the first step in achieving successful human-robot interaction in a more natural human-like way

On the Relationship between People, Objects, & Interactive Technologies: Transforming Digital & Physical experiences through the process of Realizing Empathy

La manera com les persones es relacionen amb el seu entorn, ja sigui físic o digital, és cada cop més complexa i fugaç, fent que la relació de l'usuari amb els seus objectes i eines digitals, de vegades, sigui extrema i de curta durada. Tanmateix, la propietat d'objectes i objectes tecnològics interactius no és buida de significat, són mostres de reflexió i representació per als altres i del seu paper a la societat. La clau per mantenir una relació i el significat amb aquests objectes rau en el disseny i la intenció de l'experiència interactiva creada. Inspirats en les disciplines de la psicologia, el procés de disseny, la interacció humà-ordinador i els models de negoci, aquesta tesi explora, analitza, crea i prova els fonaments teòrics sobre l'empatia i el concepte d'entaular una relació de llarga durada entre les persones i les tecnologies interactives.  Amb aquesta finalitat, aquesta tesi es divideix en 4 fases: (1) l’estudi en profunditat de les referències bibliogràfiques dins del sector HCI, amb especial atenció al rol del disseny i la psicologia amb la intenció de respondre preguntes com: “Com podem construir relacions de llarga durada entre persones i objectes intel·ligents?” (2) Recopilar i adoptar definicions, eines i terminologia de treballs relacionats que aportin a la construcció de la contribució principal d'aquesta tesi, (3) Crear i presentar un model d'interacció entre persones i tecnologia que aporti a una interacció de llarga durada, i (4) presentar un cas d’estudi on s’implementi el model proposat.  Després del treball bibliogràfic, al sector de l'HCI, s'ha identificat un buit, fruit de les principals preocupacions expressades: la manca de connexió entre la teoria i la pràctica del disseny, així com una mancança en l’àmbit de l'Empatia. El resultat fa que molts dels models d’interacció amb intenció empàtica i afectiva no se sustentin entre si. Això ens ha portat a la segona fase de la tesi on aprofitem les referències de múltiples disciplines per estudiar què és l'empatia, com s'implementa, com es percep i com evoluciona cap a l'objectiu d'una relació a llarg termini, com a punt focal cap a les principals contribucions de la tesi. .  Després de reunir i analitzar exhaustivament les referències al voltant de l'empatia, entrem a la tercera fase on presentem el model teòric d'interacció amb el potencial d'establir una interacció a llarg termini i l’anomenat Procés de realització de l'empatia (RE). Més que intentar definir què és l'empatia, aquesta proposta intenta oferir una perspectiva diferent de l'empatia i visualitza el seu abast com un procés influenciat per models de diàleg i col·laboració amb el propòsit de crear comprensió mútua i donar significat a aquest intercanvi.  Amb un model clar i una sòlida base teòrica, la fase final de la tesi cerca provar el model proposat amb l'objectiu d'observar si es poden detectar indicadors d'afecció afectiva i confiança entre una persona i el seu objecte tecnològic. En aquest cas, vam tenir l'oportunitat de treballar amb robots socials com el nostre “altre actor” per dissenyar les proves del model. Aquestes proves pretenien capturar els indicadors d'empatia entre un humà i un robot que abraça: l'aferrament afectiu, la confiança, la regulació de les expectatives i la reflexió sobre la perspectiva de l'altre dins un conjunt d'estratègies de col·laboració. Plantegem la hipòtesi que una estratègia de col·laboració activa condueix a un compromís més significatiu de generar empatia entre un humà i un robot en comparació amb una estratègia passiva. Els resultats són encoratjadors i clarament estableixen un camí per a futures investigacions sobre el disseny d'aquest model. La forma en que las personas se relacionan con su entorno, ya sea físico o digital, se vuelve cada vez más compleja y fugaz, haciendo que la relación del usuario con sus objetos y herramientas digitales, en ocasiones, sea extrema y de corta duración. Sin embargo, la propiedad de objetos y objetos tecnológicos interactivos no es vacía de significado, son muestras de reflexión y representación para los demás y de su papel en la sociedad. La clave para mantener una relación y significado con estos objetos radica en el diseño y la intención de la experiencia interactiva creada. Inspirados en las disciplinas de la psicología, el proceso de diseño, la interacción humano-ordenador y los modelos de negocio, esta tesis explora, analiza, crea y prueba los fundamentos teóricos sobre la empatía y el concepto de entablar una relación de larga duración entre las personas y las tecnologías interactivas.  Con este fin, esta tesis se divide en 4 fases: (1) estudio en profundidad de las referencias bibliográficas dentro del sector HCI, con especial atención al rol del diseño y la psicología con la intención de responder a preguntas como: “¿Cómo podemos construir relaciones de larga duración entre personas y objetos inteligentes?”(2) Recopilar y adoptar definiciones, herramientas y terminología de trabajos relacionados que aporten a la construcción de la contribución principal de esta tesis, (3) Crear y presentar un modelo de interacción entre personas y tecnología que aporte a una interacción de larga duración, y (4) presentar un caso de estudio donde se implemente el modelo propuesto.  Tras el trabajo bibliográfico en el sector del HCI se ha identificado un vacío, fruto de las principales preocupaciones expresadas: la falta de conexión entre la teoría y la práctica del diseño, así como una falta en el tema de la Empatía. El resultado hace que muchos de los modelos de interacción con intención empática y afectiva no se sustenten entre sí. Esto nos ha llevado a la segunda fase de la tesis en la que aprovechamos las referencias de múltiples disciplinas para estudiar qué es la empatía, cómo se implementa, cómo se percibe y cómo evoluciona hacia el objetivo de una relación a largo plazo, como punto focal hacia las principales contribuciones de la tesis. .  Después de una reunir y analizar exhaustivamente las referencias en torno a la empatía, entramos en la tercera fase donde presentamos el modelo teórico de interacción con el potencial de entablar una interacción a largo plazo y denominado Proceso de realización de la empatía (RE). Más que intentar definir qué es la empatía, esta propuesta trata de ofrecer una perspectiva diferente a la empatía y visualiza su alcance como un proceso influenciado por modelos de diálogo y colaboración con el propósito de crear comprensión mutua y dar significado a ese intercambio.  Con un modelo claro y una sólida base teórica, la fase final de la tesis busca probar el modelo propuesto con el objetivo de observar si el modelo puede detectar indicadores de Apego Afectivo y Confianza entre una persona y su objeto tecnológico. En el caso de este trabajo, tuvimos la oportunidad de trabajar con robots sociales como nuestro “otro actor” para diseñar las pruebas del modelo. Estas pruebas pretendían capturar los indicadores de de empatía entre un humano y un robot que abarca: el apego afectivo, la confianza, la regulación de las expectativas y la reflexión sobre la perspectiva del otro dentro de un conjunto de estrategias de colaboración. Planteamos la hipótesis de que una estrategia de colaboración activa conduce a un compromiso más significativo de generar empatía entre un humano y un robot en comparación con una estrategia pasiva. Los resultados son alentadores y claramente establecen un camino para futuras investigaciones sobre el diseño de este modelo. How people engage with their surroundings, whether physical or digital, becomes increasingly complex and rapid, making the user’s relationship with their objects and digital tools, at times, extreme and short-lived. Yet, there is still meaning in ownership of objects and interactive technological objects, they are tokens of reflection and representation to others and their role in society. The key to sustaining a relationship and sense of meaning with these objects lies in the design and intention of the interactive experience created. Inspired by disciplines of psychology, design, Human-computer interaction, and business modeling, this thesis explored, analyzed, created, and tested theoretical foundations on Empathy and the concept of initiating a long-term relationship between people and their interactive technologies.  To that end, the thesis book was managed in 4 main stages: (1) presenting a deeper dive into bibliographic references within HCI and the role of both design and psychology in the attempt to tackle questions like: “How can we build long-term relationships between people and their smart objects?” (2) Collect and adopt from related works that helped build the main contributions of the thesis book, (3) Create an interaction model between humans and their technology that lent itself for potential long-term engagement, and (4) a case study that implemented and instantiated the model designed.     After mapping the HCI bibliographical works in the first phase, a gap was revealed indicative of the main concerns expressed: a lack of connection between theory and design practice as well as a lack in the topic of Empathy. The result makes many of the models of interaction with empathetic and affective intention unsupported between each other. This has led us to the second phase of the thesis where we leveraged references across multiple disciplines to survey what empathy is, how it is implemented, perceived and evolved toward the goal of long-term relationship, as a focal point toward the main thesis contributions.   After an exhaustive gathering and analysis of the work around Empathy, we entered the third phase where we present the proposed theoretical model of interaction with the potential for long-term engagement named the Process of Realizing Empathy (RE). Rather than attempting to further define empathy, this proposal is about offering a different perspective to empathy that visualizes its scope as a process influenced by dialogue and collaborative models with the goal to reach meaning between the actors involved.  With a clear model in place and a strong theoretical foundation, the final phase of the thesis looked to test the proposed model with the goal of observing if the model can provoke its indicators of Affective Attachment and Trust between a person and their technological object. In the case of this thesis work, we had the opportunity to work with social robots as our “other actor” to design the tests for the model. This testbed meant to capture the indicators of early empathy realization between a human and a robot encompassing affective attachment, trust, expectation regulation, and reflecting on the other’s perspective within a set of collaborative strategies. We hypothesized that an active collaboration strategy is conducive to a more meaningful and purposeful engagement of realizing empathy between a human and a robot compared to a passive one. The results are encouraging and clearly establish a path for further research on this model’s design.

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

Anonymous Panda: preserving anonymity and expressiveness in online mental health platforms

Digital solutions that allow people to seek treatment, such as online psychological interventions and other technology-mediated therapies, have been developed to assist individuals with mental health disorders. Such approaches may raise privacy concerns about the use of people’s data and the safety of their mental health information. This work uses cutting-edge computer graphics technology to develop a novel system capable of increasing anonymity while maintaining expressiveness in computer-mediated mental health interventions. According to our preliminary findings, we were able to customize a realistic avatar using Live Link, Metahumans, and Unreal Engine 4 (UE4) with the same emotional depth as a real person. Furthermore, these findings showed that the virtual avatars’ inability to express themselves through hand motion gave the impression that they were acting in an unnatural way. By including the hand tracking feature using the Leap Motion Controller, we were able to improve our comprehension of the prospective use of ultra-realistic virtual human avatars in video conferencing therapy, i.e., both studies helped us understand how vital facial and body expressions are and how problematic their absence is in communicating with others.Soluções digitais que permitem às pessoas procurar tratamento, tais como terapias psicológicas online e outras terapias com recurso à tecnologia, foram desenvolvidas para ajudar indivíduos com distúrbios de saúde mental. Tais abordagens podem suscitar preocupações sobre a privacidade na utilização dos dados das pessoas e a segurança da informação sobre a sua saúde mental. Este trabalho utiliza tecnologia de ponta em computação gráfica para desenvolver um sistema inovador capaz de aumentar o anonimato, mantendo simultaneamente a expressividade nas inter venções de saúde mental mediadas por computador. Segundo os nossos resultados preliminares, conseguimos personalizar um avatar realista usando Live Link, Metahumans, e Unreal Engine 4 (UE4) com a mesma profundidade emocional que uma pessoa real. Além disso, os resultados mostraram que a incapacidade dos avatares virtuais de se expressarem através do movimento das mãos deu a impressão de que estavam a agir de uma forma pouco natural. Ao incluir a função de rastreio das mãos utilizando o Leap Motion Controller, conseguimos melhorar a nossa compreensão do uso prospetivo de avatares humanos virtuais e ultrarrealistas na terapia de videoconferência, ou seja, os estudos realizados ajudaram-nos a compreender como as expressões faciais e corporais são vitais e como a sua ausência é problemática na comunicação com os outros

Do Women Adapt More? The Gender Effects of the Speaker on Classroom Speech Evaluations

The presentational style of females and the connection with the audience members was examined. 95 male and female subjects gathered the results from five different sections of speech communication classes. The rating scale that was completed by the subjects contained traits such as, organization, language, material, delivery, analysis and voice. The results have indicated that the female presentational style does have more of a connection and adaptation with the audience than do males

Do Women Adapt More? The Gender Effects of the Speaker on Classroom Speech Evaluations

The presentational style of females and the connection with the audience members was examined. 95 male and female subjects gathered the results from five different sections of speech communication classes. The rating scale that was completed by the subjects contained traits such as, organization, language, material, delivery, analysis and voice. The results have indicated that the female presentational style does have more of a connection and adaptation with the audience than do males

Real-time generation and adaptation of social companion robot behaviors

Social robots will be part of our future homes. They will assist us in everyday tasks, entertain us, and provide helpful advice. However, the technology still faces challenges that must be overcome to equip the machine with social competencies and make it a socially intelligent and accepted housemate. An essential skill of every social robot is verbal and non-verbal communication. In contrast to voice assistants, smartphones, and smart home technology, which are already part of many people's lives today, social robots have an embodiment that raises expectations towards the machine. Their anthropomorphic or zoomorphic appearance suggests they can communicate naturally with speech, gestures, or facial expressions and understand corresponding human behaviors. In addition, robots also need to consider individual users' preferences: everybody is shaped by their culture, social norms, and life experiences, resulting in different expectations towards communication with a robot. However, robots do not have human intuition - they must be equipped with the corresponding algorithmic solutions to these problems. This thesis investigates the use of reinforcement learning to adapt the robot's verbal and non-verbal communication to the user's needs and preferences. Such non-functional adaptation of the robot's behaviors primarily aims to improve the user experience and the robot's perceived social intelligence. The literature has not yet provided a holistic view of the overall challenge: real-time adaptation requires control over the robot's multimodal behavior generation, an understanding of human feedback, and an algorithmic basis for machine learning. Thus, this thesis develops a conceptual framework for designing real-time non-functional social robot behavior adaptation with reinforcement learning. It provides a higher-level view from the system designer's perspective and guidance from the start to the end. It illustrates the process of modeling, simulating, and evaluating such adaptation processes. Specifically, it guides the integration of human feedback and social signals to equip the machine with social awareness. The conceptual framework is put into practice for several use cases, resulting in technical proofs of concept and research prototypes. They are evaluated in the lab and in in-situ studies. These approaches address typical activities in domestic environments, focussing on the robot's expression of personality, persona, politeness, and humor. Within this scope, the robot adapts its spoken utterances, prosody, and animations based on human explicit or implicit feedback.Soziale Roboter werden Teil unseres zukünftigen Zuhauses sein. Sie werden uns bei alltäglichen Aufgaben unterstützen, uns unterhalten und uns mit hilfreichen Ratschlägen versorgen. Noch gibt es allerdings technische Herausforderungen, die zunächst überwunden werden müssen, um die Maschine mit sozialen Kompetenzen auszustatten und zu einem sozial intelligenten und akzeptierten Mitbewohner zu machen. Eine wesentliche Fähigkeit eines jeden sozialen Roboters ist die verbale und nonverbale Kommunikation. Im Gegensatz zu Sprachassistenten, Smartphones und Smart-Home-Technologien, die bereits heute Teil des Lebens vieler Menschen sind, haben soziale Roboter eine Verkörperung, die Erwartungen an die Maschine weckt. Ihr anthropomorphes oder zoomorphes Aussehen legt nahe, dass sie in der Lage sind, auf natürliche Weise mit Sprache, Gestik oder Mimik zu kommunizieren, aber auch entsprechende menschliche Kommunikation zu verstehen. Darüber hinaus müssen Roboter auch die individuellen Vorlieben der Benutzer berücksichtigen. So ist jeder Mensch von seiner Kultur, sozialen Normen und eigenen Lebenserfahrungen geprägt, was zu unterschiedlichen Erwartungen an die Kommunikation mit einem Roboter führt. Roboter haben jedoch keine menschliche Intuition - sie müssen mit entsprechenden Algorithmen für diese Probleme ausgestattet werden. In dieser Arbeit wird der Einsatz von bestärkendem Lernen untersucht, um die verbale und nonverbale Kommunikation des Roboters an die Bedürfnisse und Vorlieben des Benutzers anzupassen. Eine solche nicht-funktionale Anpassung des Roboterverhaltens zielt in erster Linie darauf ab, das Benutzererlebnis und die wahrgenommene soziale Intelligenz des Roboters zu verbessern. Die Literatur bietet bisher keine ganzheitliche Sicht auf diese Herausforderung: Echtzeitanpassung erfordert die Kontrolle über die multimodale Verhaltenserzeugung des Roboters, ein Verständnis des menschlichen Feedbacks und eine algorithmische Basis für maschinelles Lernen. Daher wird in dieser Arbeit ein konzeptioneller Rahmen für die Gestaltung von nicht-funktionaler Anpassung der Kommunikation sozialer Roboter mit bestärkendem Lernen entwickelt. Er bietet eine übergeordnete Sichtweise aus der Perspektive des Systemdesigners und eine Anleitung vom Anfang bis zum Ende. Er veranschaulicht den Prozess der Modellierung, Simulation und Evaluierung solcher Anpassungsprozesse. Insbesondere wird auf die Integration von menschlichem Feedback und sozialen Signalen eingegangen, um die Maschine mit sozialem Bewusstsein auszustatten. Der konzeptionelle Rahmen wird für mehrere Anwendungsfälle in die Praxis umgesetzt, was zu technischen Konzeptnachweisen und Forschungsprototypen führt, die in Labor- und In-situ-Studien evaluiert werden. Diese Ansätze befassen sich mit typischen Aktivitäten in häuslichen Umgebungen, wobei der Schwerpunkt auf dem Ausdruck der Persönlichkeit, dem Persona, der Höflichkeit und dem Humor des Roboters liegt. In diesem Rahmen passt der Roboter seine Sprache, Prosodie, und Animationen auf Basis expliziten oder impliziten menschlichen Feedbacks an

Animating observed emotional behaviour: a practice-based investigation comparing three approaches to self-figurative animation

This research explores different animation approaches to rendering observed emotional behaviour, through the creation of an animated artefact. It opens with an introduction to the research and the methodology chosen before progressing to a review of academic and practitioner-based literature associated with observed emotional behaviour. Building upon this foundation of literature, the thesis outlines how the artifact was created with a practice based approach drawn from Haseman’s cycle of creation, feedback, reflection and then creation. The main research question is augmented by a series of contributory questions that explore the research through iterations of animation drawn from a base of live action footage of observed emotional behaviour. These exploratory iterations progress though motion capture, rotoscopy and finally freeform animation. The completed artifact and its findings are explored first though a perception study and then a production study. This thesis is based on the investigation and discourse of observed emotional behaviour surrounding the use of animation, specifically, the direct study of the observation of emotional behaviour through the application of animation as a tool of research. It aims to provide a basis of discussion and contribution to knowledge for animation practitioners, theorists and practitioner-researchers seeking to use less performative and exaggerated forms