87 research outputs found
CHARMIE: a collaborative healthcare and home service and assistant robot for elderly care
The global population is ageing at an unprecedented rate. With changes in life expectancy across the world, three major issues arise: an increasing proportion of senior citizens; cognitive and physical problems progressively affecting the elderly; and a growing number of single-person households. The available data proves the ever-increasing necessity for efficient elderly care solutions such as healthcare service and assistive robots. Additionally, such robotic solutions provide safe healthcare assistance in public health emergencies such as the SARS-CoV-2 virus (COVID-19). CHARMIE is an anthropomorphic collaborative healthcare and domestic assistant robot capable of performing generic service tasks in non-standardised healthcare and domestic environment settings. The combination of its hardware and software solutions demonstrates map building and self-localisation, safe navigation through dynamic obstacle detection and avoidance, different human-robot interaction systems, speech and hearing, pose/gesture estimation and household object manipulation. Moreover, CHARMIE performs end-to-end chores in nursing homes, domestic houses, and healthcare facilities. Some examples of these chores are to help users transport items, fall detection, tidying up rooms, user following, and set up a table. The robot can perform a wide range of chores, either independently or collaboratively. CHARMIE provides a generic robotic solution such that older people can live longer, more independent, and healthier lives.This work has been supported by FCT—Fundação para a Ciência e Tecnologia within the
R&D Units Project Scope: UIDB/00319/2020. The author T.R. received funding through a doctoral
scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência
e a Tecnologia) [grant number SFRH/BD/06944/2020], with funds from the Portuguese Ministry
of Science, Technology and Higher Education and the European Social Fund through the Programa
Operacional do Capital Humano (POCH). The author F.G. received funding through a doctoral
scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência
e a Tecnologia) [grant number SFRH/BD/145993/2019], with funds from the Portuguese Ministry
of Science, Technology and Higher Education and the European Social Fund through the Programa
Operacional do Capital Humano (POCH)
Smart rehabilitation
This thesis is born from a collaboration project between the HEIG-VD and the CHUV hospital in Lausanne, Switzerland. We study the problem of human grasp recognition from first-person RGB video input data. Grasping is the action of seizing and holding firmly an object and there exist many different types. The objective is to use grasp recognition for automating the monitoring of the rehabilitation sessions of patients with upper-limb neurological disorders. We compared three different approaches based on Deep Learning. Firstly, a naive image model that is trained with the entire images. Secondly, a video model, so apart from the spatial features it also takes advantage of the temporal dimension. Lastly, an image model that is trained with images cropped around the hands, so it focuses only on the part that determines the grasp. We used the Yale Grasping Dataset for training the models. To enhance the interpretability of the results we proposed a coarse-grained grasp grouping based on the Feix grasp taxonomy. We also captured our own small first-person video grasp dataset to test the applicability of the models to our setup, which differs from the training dataset in the camera location and angle. Considering the intrinsic challenges of the data such as the frequent hand-object occlusions or the dataset difficulties like its real-world setting and the low video quality, the results are relatively good. Nevertheless, they are insufficient for deploying a satisfactory system at the hospital and remark the difficulty of grasp recognition from just egocentric RGB data. It would be interesting to further research other data modalities such as depth data or to study the problem from the perspective of hand pose estimation and object detection. It is also clear that the field lacks a more modern and large dataset
Robust Door Operation with the Toyota Human Support Robot. Robotic perception, manipulation and learning
Robots are progressively spreading to urban, social and assistive domains. Service robots operating in domestic environments typically face a variety of objects they have to deal with to fulfill their tasks. Some of these objects are articulated such as cabinet doors and drawers. The ability to deal with such objects is relevant, as for example navigate between rooms or assist humans in their mobility. The exploration of this task rises interesting questions in some of the main robotic threads such as perception, manipulation and learning. In this work a general framework to robustly operate different types of doors with a mobile manipulator robot is proposed. To push the state-of-the-art, a novel algorithm, that fuses a Convolutional Neural Network with point cloud processing for estimating the end-effector grasping pose in real-time for multiple handles simultaneously from single RGB-D images, is proposed. Also, a Bayesian framework that embodies the robot with the ability to learn the kinematic model of the door from observations of its motion, as well as from previous experiences or human demonstrations. Combining this probabilistic approach with state-of-the-art motion planninOutgoin
Becoming Human with Humanoid
Nowadays, our expectations of robots have been significantly increases. The robot, which was initially only doing simple jobs, is now expected to be smarter and more dynamic. People want a robot that resembles a human (humanoid) has and has emotional intelligence that can perform action-reaction interactions. This book consists of two sections. The first section focuses on emotional intelligence, while the second section discusses the control of robotics. The contents of the book reveal the outcomes of research conducted by scholars in robotics fields to accommodate needs of society and industry
Class-incremental lifelong object learning for domestic robots
Traditionally, robots have been confined to settings where they operate in isolation and in highly
controlled and structured environments to execute well-defined non-varying tasks. As a result,
they usually operate without the need to perceive their surroundings or to adapt to changing
stimuli. However, as robots start to move towards human-centred environments and share the
physical space with people, there is an urgent need to endow them with the flexibility to learn
and adapt given the changing nature of the stimuli they receive and the evolving requirements
of their users. Standard machine learning is not suitable for these types of applications because
it operates under the assumption that data samples are independent and identically distributed,
and requires access to all the data in advance. If any of these assumptions is broken, the model
fails catastrophically, i.e., either it does not learn or it forgets all that was previously learned.
Therefore, different strategies are required to address this problem.
The focus of this thesis is on lifelong object learning, whereby a model is able to learn
from data that becomes available over time. In particular we address the problem of classincremental learning with an emphasis on algorithms that can enable interactive learning with
a user. In class-incremental learning, models learn from sequential data batches where each
batch can contain samples coming from ideally a single class. The emphasis on interactive
learning capabilities poses additional requirements in terms of the speed with which model
updates are performed as well as how the interaction is handled.
The work presented in this thesis can be divided into two main lines of work. First,
we propose two versions of a lifelong learning algorithm composed of a feature extractor
based on pre-trained residual networks, an array of growing self-organising networks and a
classifier. Self-organising networks are able to adapt their structure based on the input data
distribution, and learn representative prototypes of the data. These prototypes can then be
used to train a classifier. The proposed approaches are evaluated on various benchmarks under
several conditions and the results show that they outperform competing approaches in each
case. Second, we propose a robot architecture to address lifelong object learning through
interactions with a human partner using natural language. The architecture consists of an
object segmentation, tracking and preprocessing pipeline, a dialogue system, and a learning
module based on the algorithm developed in the first part of the thesis. Finally, the thesis also
includes an exploration into the contributions that different preprocessing operations have on
performance when learning from both RGB and Depth images.James Watt Scholarshi
Meta Information in Graph-based Simultaneous Localisation And Mapping
Establishing the spatial and temporal relationships between a robot, and its environment serves as a basis for scene understanding. The established approach in the literature to simultaneously build a representation of the environment, and spatially and temporally localise the robot within the environment, is Simultaneous Localisation And Mapping (SLAM). SLAM algorithms in general, and in particular visual SLAM--where the primary sensors used are cameras--have gained a great amount of attention in the robotics and computer vision communities over the last few decades due to their wide range of applications. The advances in sensing technologies and image-based learning techniques provide an opportunity to introduce additional understanding of the environment to improve the performance of SLAM algorithms.
In this thesis, I utilise meta information in a SLAM framework to achieve a robust and consistent representation of the environment and challenge some of the most limiting assumptions in the literature. I exploit structural information associated with geometric primitives, making use of the significant amount of structure present in real world scenes where SLAM algorithms are normally deployed. In particular, I exploit planarity of a group of points and introduce higher-level information associated with orthogonality and parallelism of planes to achieve structural consistency of the returned map. Separately, I also challenge the static world assumption that severely limits the deployment of autonomous mobile robotic systems in a wide range of important real world applications involving highly dynamic and unstructured environments by utilising the semantic and dynamic information in the scene. Most existing techniques try to simplify the problem by ignoring dynamics, relying on a pre-collected database of objects 3D models, imposing some motion constraints or fail to estimate the full SE(3) motions of objects in the scene which makes it infeasible to deploy these algorithms in real life scenarios of unknown and highly dynamic environments. Exploiting semantic and dynamic information in the environment allows to introduce a model-free object-aware SLAM system that is able to achieve robust moving object tracking, accurate estimation of dynamic objects full SE(3) motion, and extract velocity information of moving objects in the scene, resulting in accurate robot localisation and spatio-temporal map estimation
Human detection and face recognition in indoor environment to improve human-robot interaction in assistive and collaborative robots
Human detection in indoor environment is essential for Robots working together with humans in
collaborative manufacturing environment. Similarly, Human detection is essential for service
robots providing service with household chores or helping elderly population with different daily
activities.
Human detection can be achieved by Human Head detection, as head is the most discriminative
part of human. Head detection method can be divided into three types: i) Method based on color
mode; ii) Method based on template matching; and iii) Method based on contour detection.
Method based on color mode is simple but is error prone. Method based on head template detects
head in the image by searching for a template which is similar to head template. On the other
hand, Method based on contour detection uses some information to describe head or head and
shoulder information. The use of only one criteria may not be sufficient and accuracy of human
head detection can be increased by combining the shape and color information. In this thesis, a
method of human detection is proposed by combining the head shape and skin color (i.e.,
Combination of method based on Color mode and method based on Contour detection). Mainly,
curvature criteria is used to segment out curves having similar curvature to find human head.
Further, skin color is detected to localize face in image plane. A curve represents human head
curve if only it has sufficient skin colored pixel in its closed proximity. Thus, by using color and
human head curvature it was found that promising results could be obtained in human detection
in indoor environment.
iv
After detecting humans in the surrounding, the next step for the robot could be to identify and
recognize them. In this thesis, the use of Gabor filter response on nine points was investigated to
identify eight different individuals. This suggests that the Gabor filter on nine points could be
applied to identify people in small areas, for example home or small office with less individuals.Masters of Applied Science (M.A.Sc.) in Natural Resource Engineerin
Hybrid mapping for static and non-static indoor environments
Mención Internacional en el título de doctorIndoor environments populated by humans, such as houses, offices or universities,
involve a great complexity due to the diversity of geometries and situations that they
may present. Apart from the size of the environment, they can contain multiple rooms
distributed into floors and corridors, repetitive structures and loops, and they can
get as complicated as one can imagine. In addition, the structure and situations that
the environment present may vary over time as objects could be moved, doors can
be frequently opened or closed and places can be used for different purposes. Mobile
robots need to solve these challenging situations in order to successfully operate in
the environment. The main tools that a mobile robot has for dealing with these
situations relate to navigation and perception and comprise mapping, localization,
path planning and map adaptation. In this thesis, we try to address some of the open
problems in robot navigation in non-static indoor environments. We focus on house-like
environments as the work is framed into the HEROITEA research project that aims
attention at helping elderly people with their everyday-life activities at their homes.
This thesis contributes to HEROITEA with a complete robotic mapping system and
map adaptation that grants safe navigation and understanding of the environment.
Moreover, we provide localization and path planning strategies within the resulting
map to further operate in the environment.
The first problem tackled in this thesis is robot mapping in static indoor environments.
We propose a hybrid mapping method that structures the information gathered
from the environment into several maps. The hybrid map contains diverse knowledge of
the environment such as its structure, the navigable and blocked paths, and semantic
knowledge, such as the objects or scenes in the environment. All this information is
separated into different components of the hybrid map that are interconnected so the
system can, at any time, benefit from the information contained in every component.
In addition to the conceptual conception of the hybrid map, we have also developed
building procedures and an exploration algorithm to autonomous build the hybrid
map.
However, indoor environments populated by humans are far from being static as
the environment may change over time. For this reason, the second problem tackled in
this thesis is the adaptation of the map to non-static environments. We propose an
object-based probabilistic map adaptation that calculates the likelihood of moving or
remaining in its place for the different objects in the environment.
Finally, a map is just a description of the environment whose importance is mostly
related to how the map is used. In addition, map representations are more valuable
as long as they offer a wider range of applications. Therefore, the third problem
that we approach in this thesis is exploiting the intrinsic characteristics of the hybrid
map in order to enhance the performance of localization and path planning methods.
The particular objectives of these approaches are precision for robot localization and
efficiency for path planning in terms of execution time and traveled distance.
We evaluate our proposed methods in a diversity of simulated and real-world indoor
environments. In this extensive evaluation, we show that hybrid maps can be efficiently
built and maintained over time and they open up for new possibilities for localization
and path planning. In this thesis, we show an increase in localization precision and
robustness and an improvement in path planning performance.
In sum, this thesis makes several contributions in the context of robot navigation
in indoor environments, and especially in hybrid mapping. Hybrid maps offer higher
efficiency during map building and other applications such as localization and path
planning. In addition, we highlight the necessity of dealing with the dynamics of
indoor environments and the benefits of combining topological, semantic and metric
information to the autonomy of a mobile robot.Los entornos de interiores habitados por personas, como casas, oficinas o universidades,
entrañan una gran complejidad por la diversidad de geometrías y situaciones que pueden
ocurrir. Aparte de las diferencias en tamaño, estos entornos pueden contener muchas
habitaciones organizadas en diferentes plantas o pasillos, pueden presentar estructuras
repetitivas o bucles de tal forma que los entornos pueden llegar a ser tan complejos como
uno se pueda imaginar. Además, la estructura y el estado del entorno pueden variar
con el tiempo, ya que los objetos pueden moverse, las puertas pueden estar cerradas o
abiertas y diferentes espacios pueden ser usados para diferentes propósitos. Los robots
móviles necesitan resolver estas situaciones difíciles para poder funcionar de una forma
satisfactoria. Las principales herramientas que tiene un robot móvil para manejar
estas situaciones están relacionadas con la navegación y la percepción y comprenden el
mapeado, la localización, la planificación de trayectorias y la adaptación del mapa. En
esta tesis, abordamos algunos de los problemas sin resolver de la navegación de robots
móviles en entornos de interiores no estáticos. Nos centramos en entornos tipo casa ya
que este trabajo se enmarca en el proyecto de investigación HEROITEA que se enfoca
en ayudar a personas ancianas en tareas cotidianas del hogar. Esta tesis contribuye al
proyecto HEROITEA con un sistema completo de mapeado y adaptación del mapa
que asegura una navegación segura y la comprensión del entorno. Además, aportamos
métodos de localización y planificación de trayectorias usando el mapa construido para
realizar nuevas tareas en el entorno.
El primer problema que se aborda en esta tesis es el mapeado de entornos de
interiores estáticos por parte de un robot. Proponemos un método de mapeado híbrido
que estructura la información capturada en varios mapas. El mapa híbrido contiene
información sobre la estructura del entorno, las trayectorias libres y bloqueadas y
también incluye información semántica, como los objetos y escenas en el entorno. Toda
esta información está separada en diferentes componentes del mapa híbrido que están
interconectados de tal forma que el sistema puede beneficiarse en cualquier momento
de la información contenida en cada componente. Además de la definición conceptual del mapa híbrido, hemos desarrollado unos procedimientos para construir el mapa y un
algoritmo de exploración que permite que esta construcción se realice autónomamente.
Sin embargo, los entornos de interiores habitados por personas están lejos de ser
estáticos ya que pueden cambiar a lo largo del tiempo. Por esta razón, el segundo
problema que intentamos solucionar en esta tesis es la adaptación del mapa para
entornos no estáticos. Proponemos un método probabilístico de adaptación del mapa
basado en objetos que calcula la probabilidad de que cada objeto en el entorno haya
sido movido o permanezca en su posición anterior.
Para terminar, un mapa es simplemente una descripción del entorno cuya importancia
está principalmente relacionada con su uso. Por ello, los mapas más valiosos
serán los que ofrezcan un rango mayor de aplicaciones. Para abordar este asunto, el
tercer problema que intentamos solucionar es explotar las características intrínsecas del
mapa híbrido para mejorar el desempeño de métodos de localización y de planificación
de trayectorias usando el mapa híbrido. El objetivo principal de estos métodos es
aumentar la precisión en la localización del robot y la eficiencia en la planificación de
trayectorias en relación al tiempo de ejecución y la distancia recorrida.
Hemos evaluado los métodos propuestos en una variedad de entornos de interiores
simulados y reales. En esta extensa evaluación, mostramos que los mapas híbridos
pueden construirse y mantenerse en el tiempo de forma eficiente y que dan lugar a
nuevas posibilidades en cuanto a localización y planificación de trayectorias. En esta
tesis, mostramos un aumento en la precisión y robustez en la localización y una mejora
en el desempeño de la planificación de trayectorias.
En resumen, esta tesis lleva a cabo diversas contribuciones en el ámbito de la
navegación de robots móviles en entornos de interiores, y especialmente en mapeado
híbrido. Los mapas híbridos ofrecen más eficiencia durante la construcción del mapa
y en otras tareas como la localización y la planificación de trayectorias. Además,
resaltamos la necesidad de tratar los cambios en entornos de interiores y los beneficios
de combinar información topológica, semántica y métrica para la autonomía del robot.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Carlos Balaguer Bernaldo de Quirós.- Secretario: Javier González Jiménez.- Vocal: Nancy Marie Amat
Digital Interaction and Machine Intelligence
This book is open access, which means that you have free and unlimited access. This book presents the Proceedings of the 9th Machine Intelligence and Digital Interaction Conference. Significant progress in the development of artificial intelligence (AI) and its wider use in many interactive products are quickly transforming further areas of our life, which results in the emergence of various new social phenomena. Many countries have been making efforts to understand these phenomena and find answers on how to put the development of artificial intelligence on the right track to support the common good of people and societies. These attempts require interdisciplinary actions, covering not only science disciplines involved in the development of artificial intelligence and human-computer interaction but also close cooperation between researchers and practitioners. For this reason, the main goal of the MIDI conference held on 9-10.12.2021 as a virtual event is to integrate two, until recently, independent fields of research in computer science: broadly understood artificial intelligence and human-technology interaction
- …