Fast and Robust Detection of Fallen People from a Mobile Robot

This paper deals with the problem of detecting fallen people lying on the floor by means of a mobile robot equipped with a 3D depth sensor. In the proposed algorithm, inspired by semantic segmentation techniques, the 3D scene is over-segmented into small patches. Fallen people are then detected by means of two SVM classifiers: the first one labels each patch, while the second one captures the spatial relations between them. This novel approach showed to be robust and fast. Indeed, thanks to the use of small patches, fallen people in real cluttered scenes with objects side by side are correctly detected. Moreover, the algorithm can be executed on a mobile robot fitted with a standard laptop making it possible to exploit the 2D environmental map built by the robot and the multiple points of view obtained during the robot navigation. Additionally, this algorithm is robust to illumination changes since it does not rely on RGB data but on depth data. All the methods have been thoroughly validated on the IASLAB-RGBD Fallen Person Dataset, which is published online as a further contribution. It consists of several static and dynamic sequences with 15 different people and 2 different environments

Scene understanding for autonomous robots operating in indoor environments

Mención Internacional en el título de doctorThe idea of having robots among us is not new. Great efforts are continually made to replicate human intelligence, with the vision of having robots performing different activities, including hazardous, repetitive, and tedious tasks. Research has demonstrated that robots are good at many tasks that are hard for us, mainly in terms of precision, efficiency, and speed. However, there are some tasks that humans do without much effort that are challenging for robots. Especially robots in domestic environments are far from satisfactorily fulfilling some tasks, mainly because these environments are unstructured, cluttered, and with a variety of environmental conditions to control. This thesis addresses the problem of scene understanding in the context of autonomous robots operating in everyday human environments. Furthermore, this thesis is developed under the HEROITEA research project that aims to develop a robot system to help elderly people in domestic environments as an assistant. Our main objective is to develop different methods that allow robots to acquire more information from the environment to progressively build knowledge that allows them to improve the performance on high-level robotic tasks. In this way, scene understanding is a broad research topic, and it is considered a complex task due to the multiple sub-tasks that are involved. In that context, in this thesis, we focus on three sub-tasks: object detection, scene recognition, and semantic segmentation of the environment. Firstly, we implement methods to recognize objects considering real indoor environments. We applied machine learning techniques incorporating uncertainties and more modern techniques based on deep learning. Besides, apart from detecting objects, it is essential to comprehend the scene where they can occur. For this reason, we propose an approach for scene recognition that considers the influence of the detected objects in the prediction process. We demonstrate that the exiting objects and their relationships can improve the inference about the scene class. We also consider that a scene recognition model can benefit from the advantages of other models. We propose a multi-classifier model for scene recognition based on weighted voting schemes. The experiments carried out in real-world indoor environments demonstrate that the adequate combination of independent classifiers allows obtaining a more robust and precise model for scene recognition. Moreover, to increase the understanding of a robot about its surroundings, we propose a new division of the environment based on regions to build a useful representation of the environment. Object and scene information is integrated into a probabilistic fashion generating a semantic map of the environment containing meaningful regions within each room. The proposed system has been assessed on simulated and real-world domestic scenarios, demonstrating its ability to generate consistent environment representations. Lastly, full knowledge of the environment can enhance more complex robotic tasks; that is why in this thesis, we try to study how a complete knowledge of the environment influences the robot’s performance in high-level tasks. To do so, we select an essential task, which is searching for objects. This mundane task can be considered a precondition to perform many complex robotic tasks such as fetching and carrying, manipulation, user requirements, among others. The execution of these activities by service robots needs full knowledge of the environment to perform each task efficiently. In this thesis, we propose two searching strategies that consider prior information, semantic representation of the environment, and the relationships between known objects and the type of scene. All our developments are evaluated in simulated and real-world environments, integrated with other systems, and operating in real platforms, demonstrating their feasibility to implement in real scenarios, and in some cases outperforming other approaches. We also demonstrate how our representation of the environment can boost the performance of more complex robotic tasks compared to more standard environmental representations.La idea de tener robots entre nosotros no es nueva. Continuamente se realizan grandes esfuerzos para replicar la inteligencia humana, con la visión de tener robots que realicen diferentes actividades, incluidas tareas peligrosas, repetitivas y tediosas. La investigación ha demostrado que los robots son buenos en muchas tareas que resultan difíciles para nosotros, principalmente en términos de precisión, eficiencia y velocidad. Sin embargo, existen tareas que los humanos realizamos sin mucho esfuerzo y que son un desafío para los robots. Especialmente, los robots en entornos domésticos están lejos de cumplir satisfactoriamente algunas tareas, principalmente porque estos entornos no son estructurados, pueden estar desordenados y cuentan con una gran variedad de condiciones ambientales que controlar. Esta tesis aborda el problema de la comprensión de la escena en el contexto de robots autónomos que operan en entornos humanos cotidianos. Asimismo, esta tesis se desarrolla en el marco del proyecto de investigación HEROITEA que tiene como objetivo desarrollar un sistema robótico que funcione como asistente para ayudar a personas mayores en entornos domésticos. Nuestro principal objetivo es desarrollar diferentes métodos que permitan a los robots adquirir más información del entorno a fin de construir progresivamente un conocimiento que les permita mejorar su desempeño en tareas robóticas más complejas. En este sentido, la comprensión de escenas es un tema de investigación amplio, y se considera una tarea compleja debido a las múltiples subtareas involucradas. En esta tesis nos enfocamos específicamente en tres subtareas: detección de objetos, reconocimiento de escenas y etiquetado semántico del entorno. Por un lado, implementamos métodos para el reconocimiento de objectos considerando entornos interiores reales. Aplicamos técnicas de aprendizaje automático incorporando incertidumbres y técnicas más modernas basadas en aprendizaje profundo. Además, aparte de detectar objetos, es fundamental comprender la escena donde estos se encuentran. Por esta razón, proponemos un modelo para el reconocimiento de escenas que considera la influencia de los objetos detectados en el proceso de predicción. Demostramos que los objetos existentes y sus relaciones pueden mejorar el proceso de inferencia de la categoría de la escena. También consideramos que un modelo de reconocimiento de escenas puede beneficiarse de las ventajas de otros modelos. Por ello, proponemos un multiclasificador para el reconocimiento de escenas basado en esquemas de votación ponderados. Los experimentos llevados a cabo en entornos interiores reales demuestran que la combinación adecuada de clasificadores independientes permite obtener un modelo más robusto y preciso para el reconocimiento de escenas. Adicionalmente, para aumentar la comprensión de un robot acerca de su entorno, proponemos una nueva división del entorno basada en regiones a fin de construir una representación útil del entorno. La información de objetos y de la escena se integra de forma probabilística generando un mapa semántico que contiene regiones significativas dentro de cada habitación. El sistema propuesto ha sido evaluado en entornos domésticos simulados y reales, demostrando su capacidad para generar representaciones consistentes del entorno. Por otro lado, el conocimiento integral del entorno puede mejorar tareas robóticas más complejas; es por ello que en esta tesis analizamos cómo el conocimiento completo del entorno influye en el desempeño del robot en tareas de alto nivel. Para ello, seleccionamos una tarea fundamental, que es la búsqueda de objetos. Esta tarea mundana puede considerarse una condición previa para realizar diversas tareas robóticas complejas, como transportar objetos, tareas de manipulación, atender requerimientos del usuario, entre otras. La ejecución de estas actividades por parte de robots de servicio requiere un conocimiento profundo del entorno para realizar cada tarea de manera eficiente. En esta tesis proponemos dos estrategias de búsqueda de objetos que consideran información previa, la representación semántica del entorno, las relaciones entre los objetos conocidos y el tipo de escena. Todos nuestros desarrollos son evaluados en entornos simulados y reales, integrados con otros sistemas y operando en plataformas reales, demostrando su viabilidad de ser implementados en escenarios reales y, en algunos casos, superando a otros enfoques. También demostramos cómo nuestra representación del entorno puede mejorar el desempeño de tareas robóticas más complejas en comparación con representaciones del entorno más tradicionales.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Carlos Balaguer Bernaldo de Quirós.- Secretario: Fernando Matía Espada.- Vocal: Klaus Strob

An analysis of independent living elderly’s views on robots: A descriptive study from the norwegian context

ACHI 2020 : The Thirteenth International Conference on Advances in Computer-Human Interactions November 21 – 25, 2020This study illustrates the independent living elderly’s (≥65 years) views on robots. The data was documented through audio recordings of interviews, photos, and written logs. The analysis was done through qualitative manifest and latent content analysis. The results of the analysis were sorted into three categories: aging during the technological renaissance, domestic robots, and the elderly’s expectations of robots. The overall resulted theme was: integrating robots in the elderly’s everyday life. The results were discussed through the lenses of the Sense-of-Coherence (SOC) theoretical construct and its belonging elements: comprehensibility, manageability, and meaningfulness. The relevance of this paper contributes to giving an understanding of the domestic robots’ requirements specifications and the elderly’s expectation of human-robot interaction.publishedVersio

Federated Robust Embedded Systems: Concepts and Challenges

The development within the area of embedded systems (ESs) is moving rapidly, not least due to falling costs of computation and communication equipment. It is believed that increased communication opportunities will lead to the future ESs no longer being parts of isolated products, but rather parts of larger communities or federations of ESs, within which information is exchanged for the benefit of all participants. This vision is asserted by a number of interrelated research topics, such as the internet of things, cyber-physical systems, systems of systems, and multi-agent systems. In this work, the focus is primarily on ESs, with their specific real-time and safety requirements. While the vision of interconnected ESs is quite promising, it also brings great challenges to the development of future systems in an efficient, safe, and reliable way. In this work, a pre-study has been carried out in order to gain a better understanding about common concepts and challenges that naturally arise in federations of ESs. The work was organized around a series of workshops, with contributions from both academic participants and industrial partners with a strong experience in ES development. During the workshops, a portfolio of possible ES federation scenarios was collected, and a number of application examples were discussed more thoroughly on different abstraction levels, starting from screening the nature of interactions on the federation level and proceeding down to the implementation details within each ES. These discussions led to a better understanding of what can be expected in the future federated ESs. In this report, the discussed applications are summarized, together with their characteristics, challenges, and necessary solution elements, providing a ground for the future research within the area of communicating ESs

Robo-ethics design approach for cultural heritage: Case study - Robotics for museum purpose

The thesis shows the study behind the design process and the realization of the robotic solution for museum purposes called Virgil. The research started with the literature review on museums management and the critic analysis of signi cant digital experiences in the museum eld. Then, it continues analyzing the museum and its relation with the territory and the cultural heritage. From this preliminary analysis stage, signi cant issue related to museum management analysis comes out: nowadays many museum areas are not accessible to visitors because of issues related to security or architectural barriers. Make explorable these areas is one of the important topics in the cultural debate related to the visiting experience. This rst stage gave the knowledge to develop the outlines which brought to the realization of an ef cient service design then realized following robot ethical design values. One of the pillars of the robot ethical design is the necessity to involve all the stakeholders in the early project phases, for this reason, the second stage of the research was the study of the empathic relations between museum and visitors. In this phase, facilitator factors of this relation are de ned and transformed into guidelines for the product system performances. To perform this stage, it has been necessary create a relation between all the stakeholders of the project, which are: Politecnico di Torino, Tim (Telecom Italia Mobile) JOL CRAB research laboratory and Terre dei Savoia which is the association in charge of the Racconiggi’s Castle, the context scenario of the research. The third stage of the research, provided the realization of a prototype of the robot, in this stage telepresence robot piloted the Museum Guide it is used to show, in real time, the inaccessible areas of the museum enriched with multimedia contents. This stage concludes with the nal test user, from the test session feedback analysis, many of people want to drive themselves the robot. To give an answer to user feedback an interactive game has been developed. The game is based both on the robot ability to be driven by the visitors and also on the capacity of the robot to be used as a platform for the digital telling. To be effective, the whole experience it has been designed and tested with the support of high school students, which are one of the categories less interested in the traditional museum visit. This experience wants to demonstrate that the conscious and ethical use of the robotic device is effectively competitive, in term of performances, with the other solutions of digital visit: because it allows a more interactive digital experience in addition to the satisfaction of the physical visit at the museum

The multi-modal interface of Robot-Era multi-robot services tailored for the elderly

Socially assistive robotic platforms are now a realistic option for the long-term care of ageing populations. Elderly users may benefit from many services provided by robots operating in different environments, such as providing assistance inside apartments, serving in shared facilities of buildings or guiding people outdoors. In this paper, we present the experience gained within the EU FP7 ROBOT-ERA project towards the objective of implementing easy-to-use and acceptable service robotic system for the elderly. In particular, we detail the user-centred design and the experimental evaluation in realistic environments of a web-based multi-modal user interface tailored for elderly users of near future multi-robot services. Experimental results demonstrate positive evaluation of usability and willingness to use by elderly users, especially those less experienced with technological devices who could benefit more from the adoption of robotic services. Further analyses showed how multi-modal modes of interaction support more flexible and natural elderly–robot interaction, make clear the benefits for the users and, therefore, increase its acceptability. Finally, we provide insights and lessons learned from the extensive experimentation, which, to the best of our knowledge, is one of the largest experimentation of a multi-robot multi-service system so far

Semantic path planning for indoor navigation and household tasks

Assisting people with daily living tasks in their own homes with a robot requires a navigation through a cluttered and varying environment. Sometimes the only possible path would be blocked by an obstacle which needs to be moved away but not into other obstructing regions like the space required for opening a door. This paper presents semantic assisted path planning in which a gridded semantic map is used to improve navigation among movable obstacles (NAMO) and partially plan simple household tasks like cleaning a carpet or moving objects to another location. Semantic planning allows the execution of tasks expressed in human-like form instead of mathematical concepts like coordinates. In our numerical experiments, spatial planning was completed well within a typical human-human dialogue response time, allowing for an immediate response by the robot

Developing an Autonomous Mobile Robotic Device for Monitoring and Assisting Older People

A progressive increase of the elderly population in the world has required technological solutions capable of improving the life prospects of people suffering from senile dementias such as Alzheimer's. Socially Assistive Robotics (SAR) in the research field of elderly care is a solution that can ensure, through observation and monitoring of behaviors, their safety and improve their physical and cognitive health. A social robot can autonomously and tirelessly monitor a person daily by providing assistive tasks such as remembering to take medication and suggesting activities to keep the assisted active both physically and cognitively. However, many projects in this area have not considered the preferences, needs, personality, and cognitive profiles of older people. Moreover, other projects have developed specific robotic applications making it difficult to reuse and adapt them on other hardware devices and for other different functional contexts. This thesis presents the development of a scalable, modular, multi-tenant robotic application and its testing in real-world environments. This work is part of the UPA4SAR project ``User-centered Profiling and Adaptation for Socially Assistive Robotics''. The UPA4SAR project aimed to develop a low-cost robotic application for faster deployment among the elderly population. The architecture of the proposed robotic system is modular, robust, and scalable due to the development of functionality in microservices with event-based communication. To improve robot acceptance the functionalities, enjoyed through microservices, adapt the robot's behaviors based on the preferences and personality of the assisted person. A key part of the assistance is the monitoring of activities that are recognized through deep neural network models proposed in this work. The final experimentation of the project carried out in the homes of elderly volunteers was performed with complete autonomy of the robotic system. Daily care plans customized to the person's needs and preferences were executed. These included notification tasks to remember when to take medication, tasks to check if basic nutrition activities were accomplished, entertainment and companionship tasks with games, videos, music for cognitive and physical stimulation of the patient