Simultaneous Localization and Mapping (SLAM) for Autonomous Driving: Concept and Analysis

The Simultaneous Localization and Mapping (SLAM) technique has achieved astonishing progress over the last few decades and has generated considerable interest in the autonomous driving community. With its conceptual roots in navigation and mapping, SLAM outperforms some traditional positioning and localization techniques since it can support more reliable and robust localization, planning, and controlling to meet some key criteria for autonomous driving. In this study the authors first give an overview of the different SLAM implementation approaches and then discuss the applications of SLAM for autonomous driving with respect to different driving scenarios, vehicle system components and the characteristics of the SLAM approaches. The authors then discuss some challenging issues and current solutions when applying SLAM for autonomous driving. Some quantitative quality analysis means to evaluate the characteristics and performance of SLAM systems and to monitor the risk in SLAM estimation are reviewed. In addition, this study describes a real-world road test to demonstrate a multi-sensor-based modernized SLAM procedure for autonomous driving. The numerical results show that a high-precision 3D point cloud map can be generated by the SLAM procedure with the integration of Lidar and GNSS/INS. Online four–five cm accuracy localization solution can be achieved based on this pre-generated map and online Lidar scan matching with a tightly fused inertial system

A backpack-mounted omnidirectional camera with off-the-shelf navigation sensors for mobile terrestrial mapping: Development and forest application

The use of Personal Mobile Terrestrial System (PMTS) has increased considerably for mobile mapping applications because these systems offer dynamic data acquisition with ground perspective in places where the use of wheeled platforms is unfeasible, such as forests and indoor buildings. PMTS has become more popular with emerging technologies, such as miniaturized navigation sensors and off-the-shelf omnidirectional cameras, which enable low-cost mobile mapping approaches. However, most of these sensors have not been developed for high-accuracy metric purposes and therefore require rigorous methods of data acquisition and data processing to obtain satisfactory results for some mapping applications. To contribute to the development of light, low-cost PMTS and potential applications of these off-the-shelf sensors for forest mapping, this paper presents a low-cost PMTS approach comprising an omnidirectional camera with off-the-shelf navigation systems and its evaluation in a forest environment. Experimental assessments showed that the integrated sensor orientation approach using navigation data as the initial information can increase the trajectory accuracy, especially in covered areas. The point cloud generated with the PMTS data had accuracy consistent with the Ground Sample Distance (GSD) range of omnidirectional images (3.5–7 cm). These results are consistent with those obtained for other PMTS approaches. View Full-Text Keywords: personal mobile terrestrial system; omnidirectional cameras; low-cost sensors; forest mapping; PMTS data quality </div

Robust localization with wearable sensors

Measuring physical movements of humans and understanding human behaviour is useful in a variety of areas and disciplines. Human inertial tracking is a method that can be leveraged for monitoring complex actions that emerge from interactions between human actors and their environment. An accurate estimation of motion trajectories can support new approaches to pedestrian navigation, emergency rescue, athlete management, and medicine. However, tracking with wearable inertial sensors has several problems that need to be overcome, such as the low accuracy of consumer-grade inertial measurement units (IMUs), the error accumulation problem in long-term tracking, and the artefacts generated by movements that are less common. This thesis focusses on measuring human movements with wearable head-mounted sensors to accurately estimate the physical location of a person over time. The research consisted of (i) providing an overview of the current state of research for inertial tracking with wearable sensors, (ii) investigating the performance of new tracking algorithms that combine sensor fusion and data-driven machine learning, (iii) eliminating the effect of random head motion during tracking, (iv) creating robust long-term tracking systems with a Bayesian neural network and sequential Monte Carlo method, and (v) verifying that the system can be applied with changing modes of behaviour, defined as natural transitions from walking to running and vice versa. This research introduces a new system for inertial tracking with head-mounted sensors (which can be placed in, e.g. helmets, caps, or glasses). This technology can be used for long-term positional tracking to explore complex behaviours

Global Localization based on Evolutionary Optimization Algorithms for Indoor and Underground Environments

Mención Internacional en el título de doctorA fully autonomous robot is defined by its capability to sense, understand and move within the environment to perform a specific task. These qualities are included within the concept of navigation. However, among them, a basic transcendent one is localization, the capacity of the system to know its position regarding its surroundings. Therefore, the localization issue could be defined as searching the robot’s coordinates and rotation angles within a known environment. In this thesis, the particular case of Global Localization is addressed, when no information about the initial position is known, and the robot relies only on its sensors. This work aims to develop several tools that allow the system to locate in the two most usual geometric map representations: occupancy maps and Point Clouds. The former divides the dimensional space into equally-sized cells coded with a binary value distinguishing between free and occupied space. Point Clouds define obstacles and environment features as a sparse set of points in the space, commonly measured through a laser sensor. In this work, various algorithms are presented to search for that position through laser measurements only, in contrast with more usual methods that combine external information with motion information of the robot, odometry. Therefore, the system is capable of finding its own position in indoor environments, with no necessity of external positioning and without the influence of the uncertainty that motion sensors typically induce. Our solution is addressed by implementing various stochastic optimization algorithms or Meta-heuristics, specifically those bio-inspired or commonly known as Evolutionary Algorithms. Inspired by natural phenomena, these algorithms are based on the evolution of a series of particles or population members towards a solution through the optimization of a cost or fitness function that defines the problem. The implemented algorithms are Differential Evolution, Particle Swarm Optimization, and Invasive Weed Optimization, which try to mimic the behavior of evolution through mutation, the movement of swarms or flocks of animals, and the colonizing behavior of invasive species of plants respectively. The different implementations address the necessity to parameterize these algorithms for a wide search space as a complete three-dimensional map, with exploratory behavior and the convergence conditions that terminate the search. The process is a recursive optimum estimation search, so the solution is unknown. These implementations address the optimum localization search procedure by comparing the laser measurements from the real position with the one obtained from each candidate particle in the known map. The cost function evaluates this similarity between real and estimated measurements and, therefore, is the function that defines the problem to optimize. The common approach in localization or mapping using laser sensors is to establish the mean square error or the absolute error between laser measurements as an optimization function. In this work, a different perspective is introduced by benefiting from statistical distance or divergences, utilized to describe the similarity between probability distributions. By modeling the laser sensor as a probability distribution over the measured distance, the algorithm can benefit from the asymmetries provided by these divergences to favor or penalize different situations. Hence, how the laser scans differ and not only how much can be evaluated. The results obtained in different maps, simulated and real, prove that the Global Localization issue is successfully solved through these methods, both in position and orientation. The implementation of divergence-based weighted cost functions provides great robustness and accuracy to the localization filters and optimal response before different sources and noise levels from sensor measurements, the environment, or the presence of obstacles that are not registered in the map.Lo que define a un robot completamente autónomo es su capacidad para percibir el entorno, comprenderlo y poder desplazarse en ´el para realizar las tareas encomendadas. Estas cualidades se engloban dentro del concepto de la navegación, pero entre todas ellas la más básica y de la que dependen en buena parte el resto es la localización, la capacidad del sistema de conocer su posición respecto al entorno que lo rodea. De esta forma el problema de la localización se podría definir como la búsqueda de las coordenadas de posición y los ángulos de orientación de un robot móvil dentro de un entorno conocido. En esta tesis se aborda el caso particular de la localización global, cuando no existe información inicial alguna y el sistema depende únicamente de sus sensores. El objetivo de este trabajo es el desarrollo de varias herramientas que permitan que el sistema encuentre la localización en la que se encuentra respecto a los dos tipos de mapa más comúnmente utilizados para representar el entorno: los mapas de ocupación y las nubes de puntos. Los primeros subdividen el espacio en celdas de igual tamaño cuyo valor se define de forma binaria entre espacio libre y ocupado. Las nubes de puntos definen los obstáculos como una serie dispersa de puntos en el espacio comúnmente medidos a través de un láser. En este trabajo se presentan varios algoritmos para la búsqueda de esa posición utilizando únicamente las medidas de este sensor láser, en contraste con los métodos más habituales que combinan información externa con información propia del movimiento del robot, la odometría. De esta forma el sistema es capaz de encontrar su posición en entornos interiores sin depender de posicionamiento externo y sin verse influenciado por la deriva típica que inducen los sensores de movimiento. La solución se afronta mediante la implementación de varios tipos de algoritmos estocásticos de optimización o Meta-heurísticas, en concreto entre los denominados bio-inspirados o comúnmente conocidos como Algoritmos Evolutivos. Estos algoritmos, inspirados en varios fenómenos de la naturaleza, se basan en la evolución de una serie de partículas o población hacia una solución en base a la optimización de una función de coste que define el problema. Los algoritmos implementados en este trabajo son Differential Evolution, Particle Swarm Optimization e Invasive Weed Optimization, que tratan de imitar el comportamiento de la evolución por mutación, el movimiento de enjambres o bandas de animales y la colonización por parte de especies invasivas de plantas respectivamente. Las distintas implementaciones abordan la necesidad de parametrizar estos algoritmos para un espacio de búsqueda muy amplio como es un mapa completo, con la necesidad de que su comportamiento sea muy exploratorio, así como las condiciones de convergencia que definen el fin de la búsqueda ya que al ser un proceso recursivo de estimación la solución no es conocida. Estos algoritmos plantean la forma de buscar la localización ´optima del robot mediante la comparación de las medidas del láser en la posición real con lo esperado en la posición de cada una de esas partículas teniendo en cuenta el mapa conocido. La función de coste evalúa esa semejanza entre las medidas reales y estimadas y por tanto, es la función que define el problema. Las funciones típicamente utilizadas tanto en mapeado como localización mediante el uso de sensores láser de distancia son el error cuadrático medio o el error absoluto entre distancia estimada y real. En este trabajo se presenta una perspectiva diferente, aprovechando las distancias estadísticas o divergencias, utilizadas para establecer la semejanza entre distribuciones probabilísticas. Modelando el sensor como una distribución de probabilidad entorno a la medida aportada por el láser, se puede aprovechar la asimetría de esas divergencias para favorecer o penalizar distintas situaciones. De esta forma se evalúa como difieren las medias y no solo cuanto. Los resultados obtenidos en distintos mapas tanto simulados como reales demuestran que el problema de la localización se resuelve con éxito mediante estos métodos tanto respecto al error de estimación de la posición como de la orientación del robot. El uso de las divergencias y su implementación en una función de coste ponderada proporciona gran robustez y precisión al filtro de localización y gran respuesta ante diferentes fuentes y niveles de ruido, tanto de la propia medida del sensor, del ambiente y de obstáculos no modelados en el mapa del entorno.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Fabio Bonsignorio.- Secretario: María Dolores Blanco Rojas.- Vocal: Alberto Brunete Gonzále

Approach for reducing the computational cost of environment classification systems for mobile robots

Disertační práce se věnuje problému změny prostředí v úlohách mobilní robotiky. Zaměřuje se na využití jednodimenzionálních nevizuálních senzorů za účelem redukce výpočetních nároků. V práci je představen nový systém pro detekci a klasifikaci prostředí robota založený na datech z kamery a z nevizuálních senzorů. Nevizuální senzory zde slouží jako prostředek detekce probíhající změny, která iniciuje klasifikaci prostředí pomocí kamerových dat. To může významně snížit výpočetní nároky v porovnání se situací, kdy je zpracováván každý a nebo každý n-tý snímek obrazu. Systém je otestován na případu změny prostředí mezi vnitřním a venkovním prostředím. Přínosy této práce jsou následující: (1) Představení systému pro detekci a klasifikaci prostředí mobilního robota; (2) Analýzu state-of-the-art v oblasti Simultánní Lokalizace a Mapování za účelem zjištění otevřených problémů, které je potřeba řešit; (3) Analýza nevizuálních senzorů vzhledem k jejich vhodnosti pro danou úlohu. (4) Analýza existujících metod pro detekci změny ve 2D signálu a představení dvou jednoduchých přístupů k tomuto problému; (5) Analýza state-of-the art v oblasti klasifikace prostředí se zaměřením na klasifikaci vnitřního a venkovního prostředí; (6) Experiment porovnávající metody studované v předchozím bodu. Jedná se dle mých znalostí o nejrozsáhlejší porovnání těchto metod na jednom jediném datasetu. Navíc jsou do experimentu zahrnuty také klasifikátory založené na neuronových sítích, které dosahují lepších výsledků než klasické přístupy; (7) Vytvoření datasetu pro testování navrženého systému na sestaveném 6-ti kolovém mobilním robotu. Podle mých znalostí do této doby neexistoval dataset, který by kromě dat potřebných k řešení úlohy SLAM, naíc přidával data umožňující detekci a klasifikaci prostředí i pomocí nevizuálních dat; (8) Implementace představného systému jako open-source balík pro Robot Operating System na platformě GitHub; (9) Implementace knihovny pro výpočet globálního popisovače Centrist v C++, taktéž dostupná jako open-source na platformě GitHub.ObhájenoThis dissertation thesis deals with the problem of environment changes in the tasks of mobile robotics. In particular, it focuses on using of one-dimensional non-visual sensors in order to reduce computation cost. The work presents a new system for detection and classification of the robot environment based on data from the camera and non-visual sensors. Non-visual sensors serve as detectors of ongoing change of the environment that initiates the classification of the environment using camera data. This can significantly reduce computational demands compared to a situation where every or every n-th frame of an image is processed. The system is evaluated on the case of a change of environment between indoor and outdoor environment. The contributions of this work are the following: (1) Proposed system for detection and classification of the environment of mobile robot; (2) State-of-the-art analysis in the field of Simultaneous Localization and Mapping in order to identify existing open issues that need to be addressed; (3) Analysis of non-visual sensors with respect to their suitability for solving change detection problem. (4) Analysis of existing methods for detecting changes in 2D signal and introduction of two simple approaches to this problem; (5) State-of-the-art analysis in the field of environment classification with a focus on the classification of indoor vs. outdoor environments; (6) Experiment comparing the methods studied in the previous point. To my best knowledge, this is the most extensive comparison of these methods on a single dataset. In addition, classifiers based on neural networks, which achieve better results than classical approaches, are also included in the experiment. (7) Creation of a dataset for testing the designed system on an assembled 6-wheel mobile robot. To the best of my knowledge, there has been no dataset that, in addition to the data needed to solve the SLAM task, adds data that allows the environment to be detected and classified using non-visual data. (8) Implementation of the proposed system as an open-source package for the Robot Operating System on the GitHub platform. (9) Implementation of a library for calculating the Centrist global descriptor in C++ and Python. Library is also available as open-source on the GitHub platform

Visual-Inertial State Estimation With Information Deficiency

State estimation is an essential part of intelligent navigation and mapping systems where tracking the location of a smartphone, car, robot, or a human-worn device is required. For autonomous systems such as micro aerial vehicles and self-driving cars, it is a prerequisite for control and motion planning. For AR/VR applications, it is the first step to image rendering. Visual-inertial odometry (VIO) is the de-facto standard algorithm for embedded platforms because it lends itself to lightweight sensors and processors, and maturity in research and industrial development. Various approaches have been proposed to achieve accurate real-time tracking, and numerous open-source software and datasets are available. However, errors and outliers are common due to the complexity of visual measurement processes and environmental changes, and in practice, estimation drift is inevitable. In this thesis, we introduce the concept of information deficiency in state estimation and how to utilize this concept to develop and improve VIO systems. We look into the information deficiencies in visual-inertial state estimation, which are often present and ignored, causing system failures and drift. In particular, we investigate three critical cases of information deficiency in visual-inertial odometry: low texture environment with limited computation, monocular visual odometry, and inertial odometry. We consider these systems under three specific application settings: a lightweight quadrotor platform in autonomous flight, driving scenarios, and AR/VR headset for pedestrians. We address the challenges in each application setting and explore how the tight fusion of deep learning and model-based VIO can improve the state-of-the-art system performance and compensate for the lack of information in real-time. We identify deep learning as a key technology in tackling the information deficiencies in state estimation. We argue that developing hybrid frameworks that leverage its advantage and enable supervision for performance guarantee provides the most accurate and robust solution to state estimation

Shaped-based IMU/Camera Tightly Coupled Object-level SLAM using Rao-Blackwellized Particle Filtering

Simultaneous Localization and Mapping (SLAM) is a decades-old problem. The classical solution to this problem utilizes entities such as feature points that cannot facilitate the interactions between a robot and its environment (e.g., grabbing objects). Recent advances in deep learning have paved the way to accurately detect objects in the image under various illumination conditions and occlusions. This led to the emergence of object-level solutions to the SLAM problem. Current object-level methods depend on an initial solution using classical approaches and assume that errors are Gaussian. This research develops a standalone solution to object-level SLAM that integrates the data from a monocular camera and an IMU (available in low-end devices) using Rao Blackwellized Particle Filter (RBPF). RBPF does not assume Gaussian distribution for the error; thus, it can handle a variety of scenarios (such as when a symmetrical object with pose ambiguities is encountered). The developed method utilizes shape instead of texture; therefore, texture-less objects can be incorporated into the solution. In the particle weighing process, a new method is developed that utilizes the Intersection over the Union (IoU) area of the observed and projected boundaries of the object that does not require point-to-point correspondence. Thus, it is not prone to false data correspondences. Landmark initialization is another important challenge for object-level SLAM. In the state-of-the-art delayed initialization, the trajectory estimation only relies on the motion model provided by IMU mechanization (during the initialization), leading to large errors. In this thesis, two novel undelayed initializations are developed. One relies only on a monocular camera and IMU, and the other utilizes an ultrasonic rangefinder as well. The developed object-level SLAM is tested using wheeled robots and handheld devices, and an error (in the position) of 4.1 to 13.1 cm (0.005 to 0.028 of the total path length) has been obtained through extensive experiments using only a single object. These experiments are conducted in different indoor environments under different conditions (e.g. illumination). Further, it is shown that undelayed initialization using an ultrasonic sensor can reduce the algorithm's runtime by half

Visual Odometer on Videos of Endoscopic Capsules (VOVEC)

Desde a sua introdução em 2001, as cápsulas endoscópicas tornaram-se o principal método para obter imagens do intestino - uma região de difícil acesso através de métodos de endoscopia tradicionais - revolucionando a maneira como os diagnósticos no campo das doenças do intestino delgado são feitos. Estas cápsulas com dimensões comparáveis a um comprimido vitamínico tiram partido de uma câmera wireless para criar vídeos de 8 a 10 horas do trato digestivo dos pacientes. Devido à longa duração dos vídeos produzidos, o diagnóstico humano é moroso, entediante e propício a erros. Para além disto, depois de encontrada uma lesão, a informação da sua localização é escassa e dependente de hardware externo, levando a que uma solução baseada apensa em software com precisão melhorada seja bastante desejada. Este trabalho advém desta necessidade e, tendo-a em mente, propomos a implementação de dois métodos baseados em deep-learning, visando melhorar em relação às limitações dos sistemas atuais de localização de cápsulas endoscópicas. Para treinar e testar as nossas redes neuronais, um dataset que contém 111 vídeos da cápsula PillCam SB3 e 338 da cápsula PillCam SB2 foi utilizado, cortesia do Centro Hospitalar do Porto (CHP).O primeiro método consiste numa simples estimação do deslocamento da cápsula ao longo do intestino delgado utilizando uma HomographyNet, uma abordagem de deep-learning supervisionado usada para o cálculo de homografias entre imagens.Já no segundo método uma posição relativa 3D da cápsula é fornecida ao longo do intestino delgado, recorrendo a um método não-supervisionado de deep-learning denominado SfMLearner. Este método combina uma DepthNet e uma PoseNet para aprender a profundidade da imagem e a posição da cápsula em simultâneo.Since its introduction in 2001, capsule endoscopy has become the leading screening method for the small bowel - a region not easily accessible with traditional endoscopy techniques - revolutionizing the way diagnostics work in the field of small bowel diseases. These capsules are vitamin-sized and leverage from a small wireless camera to create 8 to 10 hour videos of the patients digestive tract. Due to the long duration of the videos produced, the human-based diagnosis is elongated, tedious and error-prone. Moreover, once a lesion is found, the localization information is scarce and hardware dependent, entailing desirability for a software-only endoscopic capsule localization system with added precision. This work stems from this need and, bearing this in mind, we propose the implementation of two deep-learning based methods to improve upon the limitations of the techniques used so far for the capsule position estimation. To train and test our networks, a dataset of 111 PillCam SB3 and 338 PillCam SB2 videos were used, courtesy of Centro Hospitalar do Porto (CHP).The first method consists in a simple capsule displacement estimation throughout the small bowel utilizing HomographyNet, a deep learning supervised approach that is used for homography computation between images. (DeTone et al. (2016))Differently, the second proposed method is intended to provide a 3D position along the small intestine, utilizing a deep learning unsupervised approach labeled SfMLearner, which takes advantage of a combination between a DepthNet and a PoseNet to learn depth and ego-motion from video simultaneously. (Zhou et al. (2017)