ON THE INFLUENCE OF SOCIAL ROBOTS IN COGNITIVE MULTITASKING AND ITS APPLICATION

[Objective] I clarify the impact of social robots on cognitive tasks, such as driving a car or driving an airplane, and show the possibility of industrial applications based on the principles of social robotics. [Approach] I adopted the MATB, a generalized version of the automobile and airplane operation tasks, as cognitive tasks to evaluate participants' performance on reaction speed, tracking performance, and short-term memory tasks that are widely applicable, rather than tasks specific to a particular situation. Also, as the stimuli from social robots, we used the iCub robot, which has been widely used in social communication research. In the analysis of participants, I not only analyzed performance, but also mental workload using skin conductance and emotional analysis of arousal-valence using facial expressions analysis. In the first experiment, I compared a social robot that use social signals with a nonsocial robot that do not use such signals and evaluated whether social robots affect cognitive task performances. In the second experiment, I focused on vitality forms and compared a calm social robot with an assertive social robot. As analysis methods, I adopted Mann-Whitney's U test for one-pair comparisons, and ART-ANOVA for analysis of variance in repeated task comparisons. Based on the results, I aimed to express vitality forms in a robot head, which is smaller in size and more flexible in placement than a full-body humanoid robot, considering car and airplane cockpit's limited space. For that, I developed a novel eyebrow and I decided to use a wire-driven technique, which is widely used in surgical robots to control soft materials. [Main results] In cognitive tasks such as car drivers and airplane pilots, I clarified the effects of social robots acting social behaviors on task performance, mental workload, and emotions. In addition, I focused on vitality forms, one of the parameters of social behaviors, and clarified the effects of different vitality forms of social robots' behavior on cognitive tasks.In cognitive tasks such as car drivers and airplane pilots, we clarified the effects of social robots acting in social behaviors on task performance, mental workload, and emotions, and showed that the presence of social robots can be effective in cognitive tasks. Furthermore, focusing on vitality forms, one of the parameters of social behaviors, we clarified the effects of different vitality forms of social robots' behaviors on cognitive tasks, and found that social robots with calm behaviors positively affected participants' facial expressions and improved their performance in a short-term memory task. Based on the results, I decided to adopt the configuration of a robot head, eliminating the torso from the social humanoid robot, iCub, considering the possibility of placement in a limited space such as cockpits of car or airplane. In designing the robot head, I developed a novel soft-material eyebrow that can be mounted on the iCub robot head to achieve continuous position and velocity changes, which is an important factor to express vitality forms. The novel eyebrows can express different vitality forms by changing the shape and velocity of the eyebrows, which was conventionally represented by the iCub's torso and arms. [Significance] The results of my research are important achievements that opens up the possibility of applying social robots to non-robotic industries such as automotive and aircraft. In addition, the newly developed soft-material eyebrows' precise shape and velocity changes have opened up new research possibilities in social robotics and social communication research themselves, enabling experiments with complex facial expressions that move beyond Ekman's simple facial expression changes definition, such as, joy, anger, sadness, and pleasure. Thus, the results of this research are one important step in both scientific and industrial applications. [Key-words] social robot, cognitive task, vitality form, robot head, facial expression, eyebro

Touch Technology in Affective Human, Robot, Virtual-Human Interactions: A Survey

Given the importance of affective touch in human interactions, technology designers are increasingly attempting to bring this modality to the core of interactive technology. Advances in haptics and touch-sensing technology have been critical to fostering interest in this area. In this survey, we review how affective touch is investigated to enhance and support the human experience with or through technology. We explore this question across three different research areas to highlight their epistemology, main findings, and the challenges that persist. First, we review affective touch technology through the human–computer interaction literature to understand how it has been applied to the mediation of human–human interaction and its roles in other human interactions particularly with oneself, augmented objects/media, and affect-aware devices. We further highlight the datasets and methods that have been investigated for automatic detection and interpretation of affective touch in this area. In addition, we discuss the modalities of affective touch expressions in both humans and technology in these interactions. Second, we separately review how affective touch has been explored in human–robot and real-human–virtual-human interactions where the technical challenges encountered and the types of experience aimed at are different. We conclude with a discussion of the gaps and challenges that emerge from the review to steer research in directions that are critical for advancing affective touch technology and recognition systems. In our discussion, we also raise ethical issues that should be considered for responsible innovation in this growing area

Cross-Subject Emotion Recognition with Sparsely-Labeled Peripheral Physiological Data Using SHAP-Explained Tree Ensembles

There are still many challenges of emotion recognition using physiological data despite the substantial progress made recently. In this paper, we attempted to address two major challenges. First, in order to deal with the sparsely-labeled physiological data, we first decomposed the raw physiological data using signal spectrum analysis, based on which we extracted both complexity and energy features. Such a procedure helped reduce noise and improve feature extraction effectiveness. Second, in order to improve the explainability of the machine learning models in emotion recognition with physiological data, we proposed Light Gradient Boosting Machine (LightGBM) and SHapley Additive exPlanations (SHAP) for emotion prediction and model explanation, respectively. The LightGBM model outperformed the eXtreme Gradient Boosting (XGBoost) model on the public Database for Emotion Analysis using Physiological signals (DEAP) with f1-scores of 0.814, 0.823, and 0.860 for binary classification of valence, arousal, and liking, respectively, with cross-subject validation using eight peripheral physiological signals. Furthermore, the SHAP model was able to identify the most important features in emotion recognition, and revealed the relationships between the predictor variables and the response variables in terms of their main effects and interaction effects. Therefore, the results of the proposed model not only had good performance using peripheral physiological data, but also gave more insights into the underlying mechanisms in recognizing emotions

Output from VIP cells of the mammalian central clock regulates daily physiological rhythms

The suprachiasmatic nucleus (SCN) circadian clock is critical for optimising daily cycles in mammalian physiology and behaviour. The roles of the various SCN cell types in communicating timing information to downstream physiological systems remain incompletely understood, however. In particular, while vasoactive intestinal polypeptide (VIP) signalling is essential for SCN function and whole animal circadian rhythmicity, the specific contributions of VIP cell output to physiological control remains uncertain. Here we reveal a key role for SCN VIP cells in central clock output. Using multielectrode recording and optogenetic manipulations, we show that VIP neurons provide coordinated daily waves of GABAergic input to target cells across the paraventricular hypothalamus and ventral thalamus, supressing their activity during the mid to late day. Using chemogenetic manipulation, we further demonstrate specific roles for this circuitry in the daily control of heart rate and corticosterone secretion, collectively establishing SCN VIP cells as influential regulators of physiological timing

A shape memory alloy-based biomimetic robotic hand : design, modelling and experimental evaluation

Every year more the 400,000 people are subject to an upper limb amputation. Projections foresee that this number may double by the 2050. Infections, trauma, cancer, or complications that arise in blood vessels represent the main causes for amputations. The access to prosthetic care is worldwide extremely limited. This is mainly due to the high cost both of commercially available prostheses and of the rehabilitation procedure which every prostheses user has to endure. Aside from high costs, commercially available hand prostheses have faced high rejection rates, mainly due to the their heavy weight, noisy operation and also to the unnatural feel of the fingers. To overcome these limitations, new materials, such as Shape Memory Alloys (SMAs), have been considered as potential candidate actuators for these kind of devices. In order to provide a contribution in the development of performant and easily affordable hand prostheses, the development of a novel and cost-effective five-fingered hand prototype actuated by Shape Memory Alloy (SMA) wires is presented in this work. The dissertation starts with the description of a first generation of a SMA actuated finger. Structure assemblage and performances in term of force, motion and reactiveness are investigated to highlight advantages and disadvantages of the prototype. In order to improve the achievable performances, a second generation of SMA actuated finger having soft features is introduced. Its structure, a five-fingered hand prosthesis having intrinsically elastic fingers, capable to grasp several types of objects with a considerable force, and an entirely 3D printed structure is then presented. Comparing this prototype with the most important prostheses developed so far, relevant advantages especially in term of noiseless actuation, cost, weight, responsiveness and force can be highlighted. A finite element based framework is then developed, to enable additional structure optimization and further improve the SMA finger performances. On the same time, a concentrated parameters physics-based model is formulated to allow, in the future, an easier control of the device, characterized by strong nonlinearities mainly due to the Shape Memory alloy hysteretic behavior.Jedes Jahr werden weltweit bei mehr als 400.000 Menschen Amputationen der oberen Gliedmaßen durchgeführt. Prognosen gehen davon aus, dass sich diese Zahl bis zum Jahr 2050 verdoppeln wird. Hauptursachen der Amputationen sind Infektionen, Unfälle, Krebs oder Durchblutungsstörungen. Der Zugang zu prothetischer Versorgung ist besonders in den Entwicklungsländern stark eingeschränkt. Dies liegt vor allem an den hohen Kosten sowohl der im Handel erhältlichen Prothesen als auch des Rehabilitationsprozesses, den jeder Prothesenträger durchlaufen muss. Neben den hohen Kosten haben kommerziell erhältliche Handprothesen aufgrund ihres hohen Gewichts, des lauten Betriebes und auch des unnatürlichen Gefühls hohe Ablehnungsraten. Um diese Einschränkungen zu überwinden, wurden neue Materialien, wie z.B. Formgedächtnislegierungen (SMAs), als potenzielle Materialien für den Antrieb von Prothesen untersucht . Um einen Beitrag zur Entwicklung von leistungsfähigen und erschwinglichen Handprothesen zu leisten, wird in dieser Arbeit die Entwicklung eines neuartigen und kostengünstigen Fünf-Finger-Handprototyps vorgestellt, der durch Drähte aus Formgedächtnislegierungen aktiviert wird. Die Doktorarbeit beginnt mit der Beschreibung der ersten Generation eines SMA-aktivierten Fingers. Zuerst wird der Aufbau und das Wirkungsprinzip des SMA Fingers erläutert und die Leistungs- und Bewegungsfähigkeit des Systems untersucht sowie Vor- und Nachteile des Prototyps dargestellt. Anschließend, um die erreichbare Leistungsfähigkeit zu verbessern, wird eine zweite Generation von SMA-gesteuerten Fingern vorgestellt, die eine vollständig in 3D gedruckte Struktur aufweisen. Diese Fünffinger-Handprothese mit inhärent elastischen Fingern ermöglicht nicht nur das Greifen unterschiedlich geformter Objekte sondern auch das Heben und Halten schwerer Gegenstände. Dieser neuartige Prototyp wird mit den wichtigsten bisher entwickelten Prothesen verglichen und die relevanten Vorteile insbesondere in Bezug auf geräuschlose Ansteuerung, Kosten, Gewicht, Reaktionszeit und Kraft hervorgehoben. Abschließend wird ein Finite-Elemente-Modell entwickelt, mit Hilfe dessen die Fingerstruktur weiter optimiert und die Leistungsfähigkeit des SMA-Fingers noch verbessert werden kann. Zusätzlich wird ein Konzentriertes-Parameter-Modell formuliert, um, in der Zukunft, eine leichtere Regelung des Systems zu ermöglichen. Dieses ist notwendig, da der SMA-Finger starke Nichtlinearitäten aufweist, die auf das hysteretische Verhalten der Formgedächtnislegierung zurückzuführen sind

Chapter From the Lab to the Real World: Affect Recognition Using Multiple Cues and Modalities

Interdisciplinary concept of dissipative soliton is unfolded in connection with ultrafast fibre lasers. The different mode-locking techniques as well as experimental realizations of dissipative soliton fibre lasers are surveyed briefly with an emphasis on their energy scalability. Basic topics of the dissipative soliton theory are elucidated in connection with concepts of energy scalability and stability. It is shown that the parametric space of dissipative soliton has reduced dimension and comparatively simple structure that simplifies the analysis and optimization of ultrafast fibre lasers. The main destabilization scenarios are described and the limits of energy scalability are connected with impact of optical turbulence and stimulated Raman scattering. The fast and slow dynamics of vector dissipative solitons are exposed

Don’t Touch That Dial: Psychological Reactance, Transparency, And User Acceptance Of Smart Thermostat Setting Changes

Automation inherently removes a certain amount of user control. If perceived as a loss of freedom, users may experience psychological reactance, which is a motivational state that can lead a person to engage in behaviors to reassert their freedom. In an online experiment, participants set up and communicated with a hypothetical smart thermostat. Participants read notifications about a change in the thermostat\u27s setting. Phrasing of notifications was altered across three dimensions: strength of authoritative language, deviation of temperature change from preferences, and whether or not the reason for the change was transparent. Authoritative language, temperatures outside the user\u27s preferences, and lack of transparency induced significantly higher levels of reactance. However, when the system presented a temperature change outside of the user\u27s preferences, reactance was mitigated and user acceptance was higher if the thermostat\u27s operations were transparent. Providing justification may be less likely to induce psychological reactance and increase user acceptance. This supports efforts to use behavioral approaches, such as demand response, to increase sustainability and limit the impacts of climate change

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing