RoboChain: A Secure Data-Sharing Framework for Human-Robot Interaction

Robots have potential to revolutionize the way we interact with the world around us. One of their largest potentials is in the domain of mobile health where they can be used to facilitate clinical interventions. However, to accomplish this, robots need to have access to our private data in order to learn from these data and improve their interaction capabilities. Furthermore, to enhance this learning process, the knowledge sharing among multiple robot units is the natural step forward. However, to date, there is no well-established framework which allows for such data sharing while preserving the privacy of the users (e.g., the hospital patients). To this end, we introduce RoboChain - the first learning framework for secure, decentralized and computationally efficient data and model sharing among multiple robot units installed at multiple sites (e.g., hospitals). RoboChain builds upon and combines the latest advances in open data access and blockchain technologies, as well as machine learning. We illustrate this framework using the example of a clinical intervention conducted in a private network of hospitals. Specifically, we lay down the system architecture that allows multiple robot units, conducting the interventions at different hospitals, to perform efficient learning without compromising the data privacy.Comment: 7 pages, 6 figure

Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).The automatic emotion recognition domain brings new methods and technologies that might be used to enhance therapy of children with autism. The paper aims at the exploration of methods and tools used to recognize emotions in children. It presents a literature review study that was performed using a systematic approach and PRISMA methodology for reporting quantitative and qualitative results. Diverse observation channels and modalities are used in the analyzed studies, including facial expressions, prosody of speech, and physiological signals. Regarding representation models, the basic emotions are the most frequently recognized, especially happiness, fear, and sadness. Both single-channel and multichannel approaches are applied, with a preference for the first one. For multimodal recognition, early fusion was the most frequently applied. SVM and neural networks were the most popular for building classifiers. Qualitative analysis revealed important clues on participant group construction and the most common combinations of modalities and methods. All channels are reported to be prone to some disturbance, and as a result, information on a specific symptoms of emotions might be temporarily or permanently unavailable. The challenges of proper stimuli, labelling methods, and the creation of open datasets were also identified.Peer reviewedFinal Published versio

Social Robots to enhance therapy and interaction for children: From the design to the implementation "in the wild"

En les últimes dues dècades els robots socials s'han convertit en un camp emergent en el qual encara hi ha molt per fer. Aquest camp requereix coneixements en mecànica, control, intel·ligència artificial, sistemes, etc., però també en psicologia, disseny, ètica, etc. El nostre grup de recerca de perfil interdisciplinari ha estat treballant en el disseny de robots socials en diferents aplicacions per a nens amb necessitats especials. L'objectiu d'aquesta tesi és investigar diferents escenaris en teràpia o educació on els robots socials podrien ser una eina útil per als nens. Es van realitzar 4 estudis amb diferents propòsits: (1) dissenyar activitats amb robòtica de LEGO per avaluar el comportament social dels nens amb trastorn de l'espectre autista (TEA) (entre companys i amb adults) i analitzar la seva efectivitat, (2) dissenyar un robot social per recuperar les funcionalitats més afectades a causa de traumatismes cranioencefàlics (TCE) en nens i veure l'eficàcia del tractament, (3) proporcionar un robot mascota per alleujar els sentiments d'ansietat, solitud i estrès en nens hospitalitzats, i (4) comprovar com un robot amb comportament social i amb una personalització versus robots sense aquestes característiques mostra diferències en termes d'interacció amb nens i, per tant, pot ajudar en l'efectivitat de diferents tractaments com hem esmentat anteriorment. Els resultats van revelar diferents resultats depenent de l'aplicació: (1) efectivitat amb la plataforma robòtica social que vam dissenyar en el tractament neuropsicològic per a aquells nens afectats per TCE, (2) eficàcia amb les activitats de robòtica de LEGO dissenyades per un grup de terapeutes en termes de millora d'habilitats socials (3) un efecte positiu entre els mediadors i facilitadors de la interacció i les relacions entre els diferents agents involucrats en el procés de la cura: pacients hospitalitzats, familiars, voluntaris i personal clínic, i (4) una interacció diferent, en termes de temps, entre els dos grups durant període de dues setmanes.En las últimas dos décadas los robots sociales se han convertido en un campo emergente en el que todavía hay mucho por hacer. Este campo requiere conocimientos en mecánica, control, inteligencia artificial, sistemas, etc., pero también en psicología, diseño, ética, etc. Nuestro grupo de investigación de perfil interdisciplinar ha estado trabajando en el diseño de robots sociales en diferentes aplicaciones para niños con necesidades especiales. El objetivo de esta tesis es investigar diferentes escenarios en terapia o educación donde los robots sociales podrían ser una herramienta útil para los niños. Se realizaron 4 estudios con diferentes propósitos: (1) diseñar actividades con robótica de LEGO para evaluar el comportamiento social de los niños con trastorno del espectro autista (TEA) (entre compañeros y con adultos) y analizar su efectividad, (2) diseñar un robot social para recuperar las funcionalidades más afectadas a causa de traumatismos craneoencefálicos (TCE) en niños y ver la eficacia del tratamiento, (3) proporcionar un robot mascota para aliviar los sentimientos de ansiedad, soledad y estrés en niños hospitalizados, y (4) comprobar como un robot con comportamiento social y con una personalización versus robots sin esas características muestra diferencias en términos de interacción con niños y, por tanto, puede ayudar en la efectividad de diferentes tratamientos como mencionamos anteriormente. Los resultados revelaron diferentes resultados dependiendo de la aplicación: (1) efectividad con la plataforma robótica social que diseñamos en el tratamiento neuropsicológico para aquellos niños afectadas por TCE, (2) eficacia con las actividades de robótica de LEGO diseñadas por un grupo de terapeutas en términos de mejora de habilidades sociales (3) un efecto positivo entre los mediadores y facilitadores de la interacción y las relaciones entre los diferentes agentes involucrados en el proceso del cuidado: pacientes hospitalizados, familiares, voluntarios y personal clínico, y (4) una interacción diferente, en términos de tiempo, entre ambos grupos en el promedio de un período de dos semanas.Over the past two decades social robots have become an emerging field where there are many things still to work on. This field not only requires knowledge in mechanics, control, artificial intelligence, systems, etc., but also in psychology, design, ethics, etc. Our multidisciplinary research group has been working on designing social robotic platforms in different applications for children with special needs. The aim of this thesis is to investigate different scenarios in therapy or education where social robots could be a useful tool for children. We ran 4 studies with different purposes: (1) to design activities with LEGO robotics to assess children with autism spectrum disorder (ASD) social behaviour (between peers and with adults) and to analyze the effectiveness, (2) to design a social robotic platform to recover the functionalities most affected by traumatic brain injuries (TBI) in children and see the effectiveness of the treatment, (3) to provide a pet robot to alleviate feelings of anxiety, loneliness and stress of long-term children inpatient and their bystanders, and (4) to verify how a robot with social behaviour and personalization verses those robots without, shows differences in terms of interaction with children and thus, helps the effectiveness of different treatments as we mention above. The results revealed different outcomes depending on the application: (1) effectiveness with the social robotic platform that we designed in neuropsychological treatment in those areas affected by TBI, (2) effectiveness with the LEGO robotics activities designed by a group of therapists in terms of improvement of the social skills and engagement, (3) a positive effect within mediators and facilitators of interaction and relationships between the different agents involved in the caring process: in-patients, relatives, volunteers and clinical staff (4) slight evidence towards a different interaction, in terms of time, between both groups in the average of a two-week period

ENGAGEMENT RECOGNITION WITHIN ROBOT-ASSISTED AUTISM THERAPY

Autism is a neurodevelopmental condition typically diagnosed in early childhood, which is characterized by challenges in using language and understanding abstract concepts, effective communication, and building social relationships. The utilization of social robots in autism therapy represents a significant area of research. An increasing number of studies explore the use of social robots as mediators between therapists and children diagnosed with autism. Assessing a child’s engagement can enhance the effectiveness of robot-assisted interventions while also providing an objective metric for later analysis. The thesis begins with a comprehensive multiple-session study involving 11 children diagnosed with autism and Attention Deficit Hyperactivity Disorder (ADHD). This study employs multi-purposeful robot activities designed to target various aspects of autism. The study yields both quantitative and qualitative findings based on four behavioural measures that were obtained from video recordings of the sessions. Statistical analysis reveals that adaptive therapy provides a longer engagement duration as compared to non-adaptive therapy sessions. Engagement is a key element in evaluating autism therapy sessions that are needed for acquiring knowledge and practising new skills necessary for social and cognitive development. With the aim to create an engagement recognition model, this research work also involves the manual labelling of collected videos to generate a QAMQOR dataset. This dataset comprises 194 therapy sessions, spanning over 48 hours of video recordings. Additionally, it includes demographic information for 34 children diagnosed with ASD. It is important to note that videos of 23 children with autism were collected from previous records. The QAMQOR dataset was evaluated using standard machine learning and deep learning approaches. However, the development of an accurate engagement recognition model remains challenging due to the unique personal characteristics of each individual with autism. In order to address this challenge and improve recognition accuracy, this PhD work also explores a data-driven model using transfer learning techniques. Our study contributes to addressing the challenges faced by machine learning in recognizing engagement among children with autism, such as diverse engagement activities, multimodal raw data, and the resources and time required for data collection. This research work contributes to the growing field of using social robots in autism therapy by illuminating an understanding of the importance of adaptive therapy and providing valuable insights into engagement recognition. The findings serve as a foundation for further advancements in personalized and effective robot-assisted interventions for individuals with autism