Adapting robot task planning to user preferences: an assistive shoe dressing example

The final publication is available at link.springer.comHealthcare robots will be the next big advance in humans’ domestic welfare, with robots able to assist elderly people and users with disabilities. However, each user has his/her own preferences, needs and abilities. Therefore, robotic assistants will need to adapt to them, behaving accordingly. Towards this goal, we propose a method to perform behavior adaptation to the user preferences, using symbolic task planning. A user model is built from the user’s answers to simple questions with a fuzzy inference system, and it is then integrated into the planning domain. We describe an adaptation method based on both the user satisfaction and the execution outcome, depending on which penalizations are applied to the planner’s rules. We demonstrate the application of the adaptation method in a simple shoe-fitting scenario, with experiments performed in a simulated user environment. The results show quick behavior adaptation, even when the user behavior changes, as well as robustness to wrong inference of the initial user model. Finally, some insights in a non-simulated world shoe-fitting setup are also provided.Peer ReviewedPostprint (author's final draft

Designing Human-Centered Collective Intelligence

Human-Centered Collective Intelligence (HCCI) is an emergent research area that seeks to bring together major research areas like machine learning, statistical modeling, information retrieval, market research, and software engineering to address challenges pertaining to deriving intelligent insights and solutions through the collaboration of several intelligent sensors, devices and data sources. An archetypal contextual CI scenario might be concerned with deriving affect-driven intelligence through multimodal emotion detection sources in a bid to determine the likability of one movie trailer over another. On the other hand, the key tenets to designing robust and evolutionary software and infrastructure architecture models to address cross-cutting quality concerns is of keen interest in the “Cloud” age of today. Some of the key quality concerns of interest in CI scenarios span the gamut of security and privacy, scalability, performance, fault-tolerance, and reliability. I present recent advances in CI system design with a focus on highlighting optimal solutions for the aforementioned cross-cutting concerns. I also describe a number of design challenges and a framework that I have determined to be critical to designing CI systems. With inspiration from machine learning, computational advertising, ubiquitous computing, and sociable robotics, this literature incorporates theories and concepts from various viewpoints to empower the collective intelligence engine, ZOEI, to discover affective state and emotional intent across multiple mediums. The discerned affective state is used in recommender systems among others to support content personalization. I dive into the design of optimal architectures that allow humans and intelligent systems to work collectively to solve complex problems. I present an evaluation of various studies that leverage the ZOEI framework to design collective intelligence

Personalization framework for adaptive robotic feeding assistance

The final publication is available at link.springer.comThe deployment of robots at home must involve robots with pre-defined skills and the capability of personalizing their behavior by non-expert users. A framework to tackle this personalization is presented and applied to an automatic feeding task. The personalization involves the caregiver providing several examples of feeding using Learning-by- Demostration, and a ProMP formalism to compute an overall trajectory and the variance along the path. Experiments show the validity of the approach in generating different feeding motions to adapt to user’s preferences, automatically extracting the relevant task parameters. The importance of the nature of the demonstrations is also assessed, and two training strategies are compared. © Springer International Publishing AG 2016.Peer ReviewedPostprint (author's final draft

Conversational affective social robots for ageing and dementia support

Socially assistive robots (SAR) hold significant potential to assist older adults and people with dementia in human engagement and clinical contexts by supporting mental health and independence at home. While SAR research has recently experienced prolific growth, long-term trust, clinical translation and patient benefit remain immature. Affective human-robot interactions are unresolved and the deployment of robots with conversational abilities is fundamental for robustness and humanrobot engagement. In this paper, we review the state of the art within the past two decades, design trends, and current applications of conversational affective SAR for ageing and dementia support. A horizon scanning of AI voice technology for healthcare, including ubiquitous smart speakers, is further introduced to address current gaps inhibiting home use. We discuss the role of user-centred approaches in the design of voice systems, including the capacity to handle communication breakdowns for effective use by target populations. We summarise the state of development in interactions using speech and natural language processing, which forms a baseline for longitudinal health monitoring and cognitive assessment. Drawing from this foundation, we identify open challenges and propose future directions to advance conversational affective social robots for: 1) user engagement, 2) deployment in real-world settings, and 3) clinical translation

A real-time human-robot interaction system based on gestures for assistive scenarios

Natural and intuitive human interaction with robotic systems is a key point to develop robots assisting people in an easy and effective way. In this paper, a Human Robot Interaction (HRI) system able to recognize gestures usually employed in human non-verbal communication is introduced, and an in-depth study of its usability is performed. The system deals with dynamic gestures such as waving or nodding which are recognized using a Dynamic Time Warping approach based on gesture specific features computed from depth maps. A static gesture consisting in pointing at an object is also recognized. The pointed location is then estimated in order to detect candidate objects the user may refer to. When the pointed object is unclear for the robot, a disambiguation procedure by means of either a verbal or gestural dialogue is performed. This skill would lead to the robot picking an object in behalf of the user, which could present difficulties to do it by itself. The overall system — which is composed by a NAO and Wifibot robots, a KinectTM v2 sensor and two laptops — is firstly evaluated in a structured lab setup. Then, a broad set of user tests has been completed, which allows to assess correct performance in terms of recognition rates, easiness of use and response times.Postprint (author's final draft

Social Robots to enhance therapy and interaction for children: From the design to the implementation "in the wild"

En les últimes dues dècades els robots socials s'han convertit en un camp emergent en el qual encara hi ha molt per fer. Aquest camp requereix coneixements en mecànica, control, intel·ligència artificial, sistemes, etc., però també en psicologia, disseny, ètica, etc. El nostre grup de recerca de perfil interdisciplinari ha estat treballant en el disseny de robots socials en diferents aplicacions per a nens amb necessitats especials. L'objectiu d'aquesta tesi és investigar diferents escenaris en teràpia o educació on els robots socials podrien ser una eina útil per als nens. Es van realitzar 4 estudis amb diferents propòsits: (1) dissenyar activitats amb robòtica de LEGO per avaluar el comportament social dels nens amb trastorn de l'espectre autista (TEA) (entre companys i amb adults) i analitzar la seva efectivitat, (2) dissenyar un robot social per recuperar les funcionalitats més afectades a causa de traumatismes cranioencefàlics (TCE) en nens i veure l'eficàcia del tractament, (3) proporcionar un robot mascota per alleujar els sentiments d'ansietat, solitud i estrès en nens hospitalitzats, i (4) comprovar com un robot amb comportament social i amb una personalització versus robots sense aquestes característiques mostra diferències en termes d'interacció amb nens i, per tant, pot ajudar en l'efectivitat de diferents tractaments com hem esmentat anteriorment. Els resultats van revelar diferents resultats depenent de l'aplicació: (1) efectivitat amb la plataforma robòtica social que vam dissenyar en el tractament neuropsicològic per a aquells nens afectats per TCE, (2) eficàcia amb les activitats de robòtica de LEGO dissenyades per un grup de terapeutes en termes de millora d'habilitats socials (3) un efecte positiu entre els mediadors i facilitadors de la interacció i les relacions entre els diferents agents involucrats en el procés de la cura: pacients hospitalitzats, familiars, voluntaris i personal clínic, i (4) una interacció diferent, en termes de temps, entre els dos grups durant període de dues setmanes.En las últimas dos décadas los robots sociales se han convertido en un campo emergente en el que todavía hay mucho por hacer. Este campo requiere conocimientos en mecánica, control, inteligencia artificial, sistemas, etc., pero también en psicología, diseño, ética, etc. Nuestro grupo de investigación de perfil interdisciplinar ha estado trabajando en el diseño de robots sociales en diferentes aplicaciones para niños con necesidades especiales. El objetivo de esta tesis es investigar diferentes escenarios en terapia o educación donde los robots sociales podrían ser una herramienta útil para los niños. Se realizaron 4 estudios con diferentes propósitos: (1) diseñar actividades con robótica de LEGO para evaluar el comportamiento social de los niños con trastorno del espectro autista (TEA) (entre compañeros y con adultos) y analizar su efectividad, (2) diseñar un robot social para recuperar las funcionalidades más afectadas a causa de traumatismos craneoencefálicos (TCE) en niños y ver la eficacia del tratamiento, (3) proporcionar un robot mascota para aliviar los sentimientos de ansiedad, soledad y estrés en niños hospitalizados, y (4) comprobar como un robot con comportamiento social y con una personalización versus robots sin esas características muestra diferencias en términos de interacción con niños y, por tanto, puede ayudar en la efectividad de diferentes tratamientos como mencionamos anteriormente. Los resultados revelaron diferentes resultados dependiendo de la aplicación: (1) efectividad con la plataforma robótica social que diseñamos en el tratamiento neuropsicológico para aquellos niños afectadas por TCE, (2) eficacia con las actividades de robótica de LEGO diseñadas por un grupo de terapeutas en términos de mejora de habilidades sociales (3) un efecto positivo entre los mediadores y facilitadores de la interacción y las relaciones entre los diferentes agentes involucrados en el proceso del cuidado: pacientes hospitalizados, familiares, voluntarios y personal clínico, y (4) una interacción diferente, en términos de tiempo, entre ambos grupos en el promedio de un período de dos semanas.Over the past two decades social robots have become an emerging field where there are many things still to work on. This field not only requires knowledge in mechanics, control, artificial intelligence, systems, etc., but also in psychology, design, ethics, etc. Our multidisciplinary research group has been working on designing social robotic platforms in different applications for children with special needs. The aim of this thesis is to investigate different scenarios in therapy or education where social robots could be a useful tool for children. We ran 4 studies with different purposes: (1) to design activities with LEGO robotics to assess children with autism spectrum disorder (ASD) social behaviour (between peers and with adults) and to analyze the effectiveness, (2) to design a social robotic platform to recover the functionalities most affected by traumatic brain injuries (TBI) in children and see the effectiveness of the treatment, (3) to provide a pet robot to alleviate feelings of anxiety, loneliness and stress of long-term children inpatient and their bystanders, and (4) to verify how a robot with social behaviour and personalization verses those robots without, shows differences in terms of interaction with children and thus, helps the effectiveness of different treatments as we mention above. The results revealed different outcomes depending on the application: (1) effectiveness with the social robotic platform that we designed in neuropsychological treatment in those areas affected by TBI, (2) effectiveness with the LEGO robotics activities designed by a group of therapists in terms of improvement of the social skills and engagement, (3) a positive effect within mediators and facilitators of interaction and relationships between the different agents involved in the caring process: in-patients, relatives, volunteers and clinical staff (4) slight evidence towards a different interaction, in terms of time, between both groups in the average of a two-week period

Towards a Cognitive Architecture for Socially Adaptive Human-Robot Interaction

People have a natural predisposition to interact in an adaptive manner with others, by instinctively changing their actions, tones and speech according to the perceived needs of their peers. Moreover, we are not only capable of registering the affective and cognitive state of our partners, but over a prolonged period of interaction we also learn which behaviours are the most appropriate and well-suited for each one of them individually. This universal trait that we share regardless of our different personalities is referred to as social adaptation (adaptability). Humans are always capable of adapting to the others although our personalities may influence the speed and efficacy of the adaptation. This means that in our everyday lives we are accustomed to partake in complex and personalized interactions with our peers. Carrying this ability to personalize to human-robot interaction (HRI) is highly desirable since it would provide user-personalized interaction, a crucial element in many HRI scenarios - interactions with older adults, assistive or rehabilitative robotics, child-robot interaction (CRI), and many others. For a social robot to be able to recreate this same kind of rich, human-like interaction, it should be aware of our needs and affective states and be capable of continuously adapting its behaviour to them. Equipping a robot with these functionalities however is not a straightforward task. A robust approach for solving this is implementing a framework for the robot supporting social awareness and adaptation. In other words, the robot needs to be equipped with the basic cognitive functionalities, which would allow the robot to learn how to select the behaviours that would maximize the pleasantness of the interaction for its peers, while being guided by an internal motivation system that would provide autonomy to its decision-making process. The goal of this research was threefold: attempt to design a cognitive architecture supporting social HRI and implement it on a robotic platform; study how an adaptive framework of this kind would function when tested in HRI studies with users; and explore how including the element of adaptability and personalization in a cognitive framework would in reality affect the users - would it bring an additional richness to the human-robot interaction as hypothesized, or would it instead only add uncertainty and unpredictability that would not be accepted by the robot`s human peers? This thesis covers the work done on developing a cognitive framework for human-robot interaction; analyzes the various challenges of implementing the cognitive functionalities, porting the framework on several robotic platforms and testing potential validation scenarios; and finally presents the user studies performed with the robotic platforms of iCub and MiRo, focused on understanding how a cognitive framework behaves in a free-form HRI context and if humans can be aware and appreciate the adaptivity of the robot. In summary, this thesis had the task of approaching the complex field of cognitive HRI and attempt to shed some light on how cognition and adaptation develop from both the human and the robot side in an HRI scenario

What facilitates consumers accepting service robots? A conceptual framework

Confronting with an increasing number of robots swarming into service industries to replace human personnel, studies regarding what drives consumers to use service robots leave to be, unfortunately, still fragmented. Motivated by this, based on a content analysis of the existing studies, this paper establishes a conceptual framework to comprehend the current literature for in-depth understanding concerning customer attitude and their intention to use service robots. Drawing upon a triangulation of perspectives on end-user (i.e., technology user, consumer, and network member) in adoption research, this framework adopts technology acceptance theories, service quality, and expectancy-value theory to set up the skeleton. Furthermore, the antecedents impacting customer acceptance of service robots are subdivided into robot-design, consumer-oriented, relational components, as well as exogenous factors. This paper not only elaborates on the present situation of service robot acceptance research but also promotes it by developing a comprehensive framework regarding the effect factors