Topomap: Topological Mapping and Navigation Based on Visual SLAM Maps

Visual robot navigation within large-scale, semi-structured environments deals with various challenges such as computation intensive path planning algorithms or insufficient knowledge about traversable spaces. Moreover, many state-of-the-art navigation approaches only operate locally instead of gaining a more conceptual understanding of the planning objective. This limits the complexity of tasks a robot can accomplish and makes it harder to deal with uncertainties that are present in the context of real-time robotics applications. In this work, we present Topomap, a framework which simplifies the navigation task by providing a map to the robot which is tailored for path planning use. This novel approach transforms a sparse feature-based map from a visual Simultaneous Localization And Mapping (SLAM) system into a three-dimensional topological map. This is done in two steps. First, we extract occupancy information directly from the noisy sparse point cloud. Then, we create a set of convex free-space clusters, which are the vertices of the topological map. We show that this representation improves the efficiency of global planning, and we provide a complete derivation of our algorithm. Planning experiments on real world datasets demonstrate that we achieve similar performance as RRT* with significantly lower computation times and storage requirements. Finally, we test our algorithm on a mobile robotic platform to prove its advantages.Comment: 8 page

Hybrid mapping for static and non-static indoor environments

Mención Internacional en el título de doctorIndoor environments populated by humans, such as houses, offices or universities, involve a great complexity due to the diversity of geometries and situations that they may present. Apart from the size of the environment, they can contain multiple rooms distributed into floors and corridors, repetitive structures and loops, and they can get as complicated as one can imagine. In addition, the structure and situations that the environment present may vary over time as objects could be moved, doors can be frequently opened or closed and places can be used for different purposes. Mobile robots need to solve these challenging situations in order to successfully operate in the environment. The main tools that a mobile robot has for dealing with these situations relate to navigation and perception and comprise mapping, localization, path planning and map adaptation. In this thesis, we try to address some of the open problems in robot navigation in non-static indoor environments. We focus on house-like environments as the work is framed into the HEROITEA research project that aims attention at helping elderly people with their everyday-life activities at their homes. This thesis contributes to HEROITEA with a complete robotic mapping system and map adaptation that grants safe navigation and understanding of the environment. Moreover, we provide localization and path planning strategies within the resulting map to further operate in the environment. The first problem tackled in this thesis is robot mapping in static indoor environments. We propose a hybrid mapping method that structures the information gathered from the environment into several maps. The hybrid map contains diverse knowledge of the environment such as its structure, the navigable and blocked paths, and semantic knowledge, such as the objects or scenes in the environment. All this information is separated into different components of the hybrid map that are interconnected so the system can, at any time, benefit from the information contained in every component. In addition to the conceptual conception of the hybrid map, we have also developed building procedures and an exploration algorithm to autonomous build the hybrid map. However, indoor environments populated by humans are far from being static as the environment may change over time. For this reason, the second problem tackled in this thesis is the adaptation of the map to non-static environments. We propose an object-based probabilistic map adaptation that calculates the likelihood of moving or remaining in its place for the different objects in the environment. Finally, a map is just a description of the environment whose importance is mostly related to how the map is used. In addition, map representations are more valuable as long as they offer a wider range of applications. Therefore, the third problem that we approach in this thesis is exploiting the intrinsic characteristics of the hybrid map in order to enhance the performance of localization and path planning methods. The particular objectives of these approaches are precision for robot localization and efficiency for path planning in terms of execution time and traveled distance. We evaluate our proposed methods in a diversity of simulated and real-world indoor environments. In this extensive evaluation, we show that hybrid maps can be efficiently built and maintained over time and they open up for new possibilities for localization and path planning. In this thesis, we show an increase in localization precision and robustness and an improvement in path planning performance. In sum, this thesis makes several contributions in the context of robot navigation in indoor environments, and especially in hybrid mapping. Hybrid maps offer higher efficiency during map building and other applications such as localization and path planning. In addition, we highlight the necessity of dealing with the dynamics of indoor environments and the benefits of combining topological, semantic and metric information to the autonomy of a mobile robot.Los entornos de interiores habitados por personas, como casas, oficinas o universidades, entrañan una gran complejidad por la diversidad de geometrías y situaciones que pueden ocurrir. Aparte de las diferencias en tamaño, estos entornos pueden contener muchas habitaciones organizadas en diferentes plantas o pasillos, pueden presentar estructuras repetitivas o bucles de tal forma que los entornos pueden llegar a ser tan complejos como uno se pueda imaginar. Además, la estructura y el estado del entorno pueden variar con el tiempo, ya que los objetos pueden moverse, las puertas pueden estar cerradas o abiertas y diferentes espacios pueden ser usados para diferentes propósitos. Los robots móviles necesitan resolver estas situaciones difíciles para poder funcionar de una forma satisfactoria. Las principales herramientas que tiene un robot móvil para manejar estas situaciones están relacionadas con la navegación y la percepción y comprenden el mapeado, la localización, la planificación de trayectorias y la adaptación del mapa. En esta tesis, abordamos algunos de los problemas sin resolver de la navegación de robots móviles en entornos de interiores no estáticos. Nos centramos en entornos tipo casa ya que este trabajo se enmarca en el proyecto de investigación HEROITEA que se enfoca en ayudar a personas ancianas en tareas cotidianas del hogar. Esta tesis contribuye al proyecto HEROITEA con un sistema completo de mapeado y adaptación del mapa que asegura una navegación segura y la comprensión del entorno. Además, aportamos métodos de localización y planificación de trayectorias usando el mapa construido para realizar nuevas tareas en el entorno. El primer problema que se aborda en esta tesis es el mapeado de entornos de interiores estáticos por parte de un robot. Proponemos un método de mapeado híbrido que estructura la información capturada en varios mapas. El mapa híbrido contiene información sobre la estructura del entorno, las trayectorias libres y bloqueadas y también incluye información semántica, como los objetos y escenas en el entorno. Toda esta información está separada en diferentes componentes del mapa híbrido que están interconectados de tal forma que el sistema puede beneficiarse en cualquier momento de la información contenida en cada componente. Además de la definición conceptual del mapa híbrido, hemos desarrollado unos procedimientos para construir el mapa y un algoritmo de exploración que permite que esta construcción se realice autónomamente. Sin embargo, los entornos de interiores habitados por personas están lejos de ser estáticos ya que pueden cambiar a lo largo del tiempo. Por esta razón, el segundo problema que intentamos solucionar en esta tesis es la adaptación del mapa para entornos no estáticos. Proponemos un método probabilístico de adaptación del mapa basado en objetos que calcula la probabilidad de que cada objeto en el entorno haya sido movido o permanezca en su posición anterior. Para terminar, un mapa es simplemente una descripción del entorno cuya importancia está principalmente relacionada con su uso. Por ello, los mapas más valiosos serán los que ofrezcan un rango mayor de aplicaciones. Para abordar este asunto, el tercer problema que intentamos solucionar es explotar las características intrínsecas del mapa híbrido para mejorar el desempeño de métodos de localización y de planificación de trayectorias usando el mapa híbrido. El objetivo principal de estos métodos es aumentar la precisión en la localización del robot y la eficiencia en la planificación de trayectorias en relación al tiempo de ejecución y la distancia recorrida. Hemos evaluado los métodos propuestos en una variedad de entornos de interiores simulados y reales. En esta extensa evaluación, mostramos que los mapas híbridos pueden construirse y mantenerse en el tiempo de forma eficiente y que dan lugar a nuevas posibilidades en cuanto a localización y planificación de trayectorias. En esta tesis, mostramos un aumento en la precisión y robustez en la localización y una mejora en el desempeño de la planificación de trayectorias. En resumen, esta tesis lleva a cabo diversas contribuciones en el ámbito de la navegación de robots móviles en entornos de interiores, y especialmente en mapeado híbrido. Los mapas híbridos ofrecen más eficiencia durante la construcción del mapa y en otras tareas como la localización y la planificación de trayectorias. Además, resaltamos la necesidad de tratar los cambios en entornos de interiores y los beneficios de combinar información topológica, semántica y métrica para la autonomía del robot.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Carlos Balaguer Bernaldo de Quirós.- Secretario: Javier González Jiménez.- Vocal: Nancy Marie Amat

Learning cognitive maps: Finding useful structure in an uncertain world

In this chapter we will describe the central mechanisms that influence how people learn about large-scale space. We will focus particularly on how these mechanisms enable people to effectively cope with both the uncertainty inherent in a constantly changing world and also with the high information content of natural environments. The major lessons are that humans get by with a less is more approach to building structure, and that they are able to quickly adapt to environmental changes thanks to a range of general purpose mechanisms. By looking at abstract principles, instead of concrete implementation details, it is shown that the study of human learning can provide valuable lessons for robotics. Finally, these issues are discussed in the context of an implementation on a mobile robot. © 2007 Springer-Verlag Berlin Heidelberg

Learning spatial-semantic representations from natural language descriptions and scene classifications

Abstract — We describe a semantic mapping algorithm that learns human-centric environment models from by interpreting natural language utterances. Underlying the approach is a coupled metric, topological, and semantic representation of the environment that enables the method to infer and fuse information from natural language descriptions with low-level metric and appearance data. We extend earlier work with a novel formulation incorporates spatial layout into a topological representation of the environment. We also describe a factor graph formulation of the semantic properties that encodes human-centric concepts such as type and colloquial name for each mapped region. The algorithm infers these properties by combining the user’s natural language descriptions with image- and laser-based scene classification. We also propose a mechanism to more effectively ground natural language descriptions of spatially non-local regions using semantic cues from other modalities. We describe how the algorithm employs this learned semantic information to propose valid topological hypotheses, leading to more accurate topological and metric maps. We demonstrate that integrating language with other sensor data increases the accuracy of the achieved spatial-semantic representation of the environment. I

Low-Cost Multiple-MAV SLAM Using Open Source Software

We demonstrate a multiple micro aerial vehicle (MAV) system capable of supporting autonomous exploration and navigation in unknown environments using only a sensor commonly found in low-cost, commercially available MAVs—a front-facing monocular camera. We adapt a popular open source monocular SLAM library, ORB-SLAM, to support multiple inputs and present a system capable of effective cross-map alignment that can be theoretically generalized for use with other monocular SLAM libraries. Using our system, a single central ground control station is capable of supporting up to five MAVs simultaneously without a loss in mapping quality as compared to single-MAV ORB-SLAM. We conduct testing using both benchmark datasets and real-world trials to demonstrate the capability and real-time effectiveness

Information-theoretic environment modeling for mobile robot localization

To enhance robotic computational efficiency without degenerating accuracy, it is imperative to fit the right and exact amount of information in its simplest form to the investigated task. This thesis conforms to this reasoning in environment model building and robot localization. It puts forth an approach towards building maps and localizing a mobile robot efficiently with respect to unknown, unstructured and moderately dynamic environments. For this, the environment is modeled on an information-theoretic basis, more specifically in terms of its transmission property. Subsequently, the presented environment model, which does not specifically adhere to classical geometric modeling, succeeds in solving the environment disambiguation effectively. The proposed solution lays out a two-level hierarchical structure for localization. The structure makes use of extracted features, which are stored in two different resolutions in a single hybrid feature-map. This enables dual coarse-topological and fine-geometric localization modalities. The first level in the hierarchy describes the environment topologically, where a defined set of places is described by a probabilistic feature representation. A conditional entropy-based criterion is proposed to quantify the transinformation between the feature and the place domains. This criterion provides a double benefit of pruning the large dimensional feature space, and at the same time selecting the best discriminative features that overcome environment aliasing problems. Features with the highest transinformation are filtered and compressed to form a coarse resolution feature-map (codebook). Localization at this level is conducted through place matching. In the second level of the hierarchy, the map is viewed in high-resolution, as consisting of non-compressed entropy-processed features. These features are additionally tagged with their position information. Given the identified topological place provided by the first level, fine localization corresponding to the second level is executed using feature triangulation. To enhance the triangulation accuracy, redundant features are used and two metric evaluating criteria are employ-ed; one for dynamic features and mismatches detection, and another for feature selection. The proposed approach and methods have been tested in realistic indoor environments using a vision sensor and the Scale Invariant Feature Transform local feature extraction. Through experiments, it is demonstrated that an information-theoretic modeling approach is highly efficient in attaining combined accuracy and computational efficiency performances for localization. It has also been proven that the approach is capable of modeling environments with a high degree of unstructuredness, perceptual aliasing, and dynamic variations (illumination conditions; scene dynamics). The merit of employing this modeling type is that environment features are evaluated quantitatively, while at the same time qualitative conclusions are generated about feature selection and performance in a robot localization task. In this way, the accuracy of localization can be adapted in accordance with the available resources. The experimental results also show that the hybrid topological-metric map provides sufficient information to localize a mobile robot on two scales, independent of the robot motion model. The codebook exhibits fast and accurate topological localization at significant compression ratios. The hierarchical localization framework demonstrates robustness and optimized space and time complexities. This, in turn, provides scalability to large environments application and real-time employment adequacies

Visual Perception For Robotic Spatial Understanding

Humans understand the world through vision without much effort. We perceive the structure, objects, and people in the environment and pay little direct attention to most of it, until it becomes useful. Intelligent systems, especially mobile robots, have no such biologically engineered vision mechanism to take for granted. In contrast, we must devise algorithmic methods of taking raw sensor data and converting it to something useful very quickly. Vision is such a necessary part of building a robot or any intelligent system that is meant to interact with the world that it is somewhat surprising we don\u27t have off-the-shelf libraries for this capability. Why is this? The simple answer is that the problem is extremely difficult. There has been progress, but the current state of the art is impressive and depressing at the same time. We now have neural networks that can recognize many objects in 2D images, in some cases performing better than a human. Some algorithms can also provide bounding boxes or pixel-level masks to localize the object. We have visual odometry and mapping algorithms that can build reasonably detailed maps over long distances with the right hardware and conditions. On the other hand, we have robots with many sensors and no efficient way to compute their relative extrinsic poses for integrating the data in a single frame. The same networks that produce good object segmentations and labels in a controlled benchmark still miss obvious objects in the real world and have no mechanism for learning on the fly while the robot is exploring. Finally, while we can detect pose for very specific objects, we don\u27t yet have a mechanism that detects pose that generalizes well over categories or that can describe new objects efficiently. We contribute algorithms in four of the areas mentioned above. First, we describe a practical and effective system for calibrating many sensors on a robot with up to 3 different modalities. Second, we present our approach to visual odometry and mapping that exploits the unique capabilities of RGB-D sensors to efficiently build detailed representations of an environment. Third, we describe a 3-D over-segmentation technique that utilizes the models and ego-motion output in the previous step to generate temporally consistent segmentations with camera motion. Finally, we develop a synthesized dataset of chair objects with part labels and investigate the influence of parts on RGB-D based object pose recognition using a novel network architecture we call PartNet

High-level environment representations for mobile robots

In most robotic applications we are faced with the problem of building a digital representation of the environment that allows the robot to autonomously complete its tasks. This internal representation can be used by the robot to plan a motion trajectory for its mobile base and/or end-effector. For most man-made environments we do not have a digital representation or it is inaccurate. Thus, the robot must have the capability of building it autonomously. This is done by integrating into an internal data structure incoming sensor measurements. For this purpose, a common solution consists in solving the Simultaneous Localization and Mapping (SLAM) problem. The map obtained by solving a SLAM problem is called ``metric'' and it describes the geometric structure of the environment. A metric map is typically made up of low-level primitives (like points or voxels). This means that even though it represents the shape of the objects in the robot workspace it lacks the information of which object a surface belongs to. Having an object-level representation of the environment has the advantage of augmenting the set of possible tasks that a robot may accomplish. To this end, in this thesis we focus on two aspects. We propose a formalism to represent in a uniform manner 3D scenes consisting of different geometric primitives, including points, lines and planes. Consequently, we derive a local registration and a global optimization algorithm that can exploit this representation for robust estimation. Furthermore, we present a Semantic Mapping system capable of building an \textit{object-based} map that can be used for complex task planning and execution. Our system exploits effective reconstruction and recognition techniques that require no a-priori information about the environment and can be used under general conditions

Modeling the environment with egocentric vision systems

Cada vez más sistemas autónomos, ya sean robots o sistemas de asistencia, están presentes en nuestro día a día. Este tipo de sistemas interactúan y se relacionan con su entorno y para ello necesitan un modelo de dicho entorno. En función de las tareas que deben realizar, la información o el detalle necesario del modelo varía. Desde detallados modelos 3D para sistemas de navegación autónomos, a modelos semánticos que incluyen información importante para el usuario como el tipo de área o qué objetos están presentes. La creación de estos modelos se realiza a través de las lecturas de los distintos sensores disponibles en el sistema. Actualmente, gracias a su pequeño tamaño, bajo precio y la gran información que son capaces de capturar, las cámaras son sensores incluidos en todos los sistemas autónomos. El objetivo de esta tesis es el desarrollar y estudiar nuevos métodos para la creación de modelos del entorno a distintos niveles semánticos y con distintos niveles de precisión. Dos puntos importantes caracterizan el trabajo desarrollado en esta tesis: - El uso de cámaras con punto de vista egocéntrico o en primera persona ya sea en un robot o en un sistema portado por el usuario (wearable). En este tipo de sistemas, las cámaras son solidarias al sistema móvil sobre el que van montadas. En los últimos años han aparecido muchos sistemas de visión wearables, utilizados para multitud de aplicaciones, desde ocio hasta asistencia de personas. - El uso de sistemas de visión omnidireccional, que se distinguen por su gran campo de visión, incluyendo mucha más información en cada imagen que las cámara convencionales. Sin embargo plantean nuevas dificultades debido a distorsiones y modelos de proyección más complejos. Esta tesis estudia distintos tipos de modelos del entorno: - Modelos métricos: el objetivo de estos modelos es crear representaciones detalladas del entorno en las que localizar con precisión el sistema autónomo. Ésta tesis se centra en la adaptación de estos modelos al uso de visión omnidireccional, lo que permite capturar más información en cada imagen y mejorar los resultados en la localización. - Modelos topológicos: estos modelos estructuran el entorno en nodos conectados por arcos. Esta representación tiene menos precisión que la métrica, sin embargo, presenta un nivel de abstracción mayor y puede modelar el entorno con más riqueza. %, por ejemplo incluyendo el tipo de área de cada nodo, la localización de objetos importantes o el tipo de conexión entre los distintos nodos. Esta tesis se centra en la creación de modelos topológicos con información adicional sobre el tipo de área de cada nodo y conexión (pasillo, habitación, puertas, escaleras...). - Modelos semánticos: este trabajo también contribuye en la creación de nuevos modelos semánticos, más enfocados a la creación de modelos para aplicaciones en las que el sistema interactúa o asiste a una persona. Este tipo de modelos representan el entorno a través de conceptos cercanos a los usados por las personas. En particular, esta tesis desarrolla técnicas para obtener y propagar información semántica del entorno en secuencias de imágen