Semi-Dense Visual Odometry for RGB-D Cameras Using Approximate Nearest Neighbour Fields

This paper presents a robust and efficient semidense visual odometry solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline which estimates the pose of the sensor by registering the projection of a 3D semidense map of a reference frame with the 2D semi-dense region extracted in the current frame. The processing is speeded up by efficiently implemented approximate nearest neighbour fields under the Euclidean distance criterion, which permits the use of compact Gauss-Newton updates in the optimization. The registration is formulated as a maximum a posterior problem to deal with outliers and sensor noise, and the equivalent weighted least squares problem is consequently solved by iteratively reweighted least squares method. A variety of robust weight functions are tested and the optimum is determined based on the probabilistic characteristics of the sensor model. Extensive evaluation on publicly available RGB-D datasets shows that the proposed method predominantly outperforms existing state-of-the-art methods.The work is furthermore supported by ARC grants DE150101365. Yi Zhou acknowledges the financial support from the China Scholarship Council for his PhD Scholarship No.20140602009

Exploiting Structural Regularities and Beyond: Vision-based Localization and Mapping in Man-Made Environments

Image-based estimation of camera motion, known as visual odometry (VO), plays a very important role in many robotic applications such as control and navigation of unmanned mobile robots, especially when no external navigation reference signal is available. The core problem of VO is the estimation of the camera’s ego-motion (i.e. tracking) either between successive frames, namely relative pose estimation, or with respect to a global map, namely absolute pose estimation. This thesis aims to develop efficient, accurate and robust VO solutions by taking advantage of structural regularities in man-made environments, such as piece-wise planar structures, Manhattan World and more generally, contours and edges. Furthermore, to handle challenging scenarios that are beyond the limits of classical sensor based VO solutions, we investigate a recently emerging sensor — the event camera and study on event-based mapping — one of the key problems in the event-based VO/SLAM. The main achievements are summarized as follows. First, we revisit an old topic on relative pose estimation: accurately and robustly estimating the fundamental matrix given a collection of independently estimated homograhies. Three classical methods are reviewed and then we show a simple but nontrivial two-step normalization within the direct linear method that achieves similar performance to the less attractive and more computationally intensive hallucinated points based method. Second, an efficient 3D rotation estimation algorithm for depth cameras in piece-wise planar environments is presented. It shows that by using surface normal vectors as an input, planar modes in the corresponding density distribution function can be discovered and continuously tracked using efficient non-parametric estimation techniques. The relative rotation can be estimated by registering entire bundles of planar modes by using robust L1-norm minimization. Third, an efficient alternative to the iterative closest point algorithm for real-time tracking of modern depth cameras in ManhattanWorlds is developed. We exploit the common orthogonal structure of man-made environments in order to decouple the estimation of the rotation and the three degrees of freedom of the translation. The derived camera orientation is absolute and thus free of long-term drift, which in turn benefits the accuracy of the translation estimation as well. Fourth, we look into a more general structural regularity—edges. A real-time VO system that uses Canny edges is proposed for RGB-D cameras. Two novel alternatives to classical distance transforms are developed with great properties that significantly improve the classical Euclidean distance field based methods in terms of efficiency, accuracy and robustness. Finally, to deal with challenging scenarios that go beyond what standard RGB/RGB-D cameras can handle, we investigate the recently emerging event camera and focus on the problem of 3D reconstruction from data captured by a stereo event-camera rig moving in a static scene, such as in the context of stereo Simultaneous Localization and Mapping

A multisensor SLAM for dense maps of large scale environments under poor lighting conditions

This thesis describes the development and implementation of a multisensor large scale autonomous mapping system for surveying tasks in underground mines. The hazardous nature of the underground mining industry has resulted in a push towards autonomous solutions to the most dangerous operations, including surveying tasks. Many existing autonomous mapping techniques rely on approaches to the Simultaneous Localization and Mapping (SLAM) problem which are not suited to the extreme characteristics of active underground mining environments. Our proposed multisensor system has been designed from the outset to address the unique challenges associated with underground SLAM. The robustness, self-containment and portability of the system maximize the potential applications.The multisensor mapping solution proposed as a result of this work is based on a fusion of omnidirectional bearing-only vision-based localization and 3D laser point cloud registration. By combining these two SLAM techniques it is possible to achieve some of the advantages of both approaches – the real-time attributes of vision-based SLAM and the dense, high precision maps obtained through 3D lasers. The result is a viable autonomous mapping solution suitable for application in challenging underground mining environments.A further improvement to the robustness of the proposed multisensor SLAM system is a consequence of incorporating colour information into vision-based localization. Underground mining environments are often dominated by dynamic sources of illumination which can cause inconsistent feature motion during localization. Colour information is utilized to identify and remove features resulting from illumination artefacts and to improve the monochrome based feature matching between frames.Finally, the proposed multisensor mapping system is implemented and evaluated in both above ground and underground scenarios. The resulting large scale maps contained a maximum offset error of ±30mm for mapping tasks with lengths over 100m

Distributed Robotic Vision for Calibration, Localisation, and Mapping

This dissertation explores distributed algorithms for calibration, localisation, and mapping in the context of a multi-robot network equipped with cameras and onboard processing, comparing against centralised alternatives where all data is transmitted to a singular external node on which processing occurs. With the rise of large-scale camera networks, and as low-cost on-board processing becomes increasingly feasible in robotics networks, distributed algorithms are becoming important for robustness and scalability. Standard solutions to multi-camera computer vision require the data from all nodes to be processed at a central node which represents a significant single point of failure and incurs infeasible communication costs. Distributed solutions solve these issues by spreading the work over the entire network, operating only on local calculations and direct communication with nearby neighbours. This research considers a framework for a distributed robotic vision platform for calibration, localisation, mapping tasks where three main stages are identified: an initialisation stage where calibration and localisation are performed in a distributed manner, a local tracking stage where visual odometry is performed without inter-robot communication, and a global mapping stage where global alignment and optimisation strategies are applied. In consideration of this framework, this research investigates how algorithms can be developed to produce fundamentally distributed solutions, designed to minimise computational complexity whilst maintaining excellent performance, and designed to operate effectively in the long term. Therefore, three primary objectives are sought aligning with these three stages

LIMO-Velo: A real-time, robust, centimeter-accurate estimator for vehicle localization and mapping under racing velocities

Treballs recents sobre localització de vehicles i mapeig dels seus entorns es desenvolupen per a dispositius portàtils o robots terrestres que assumeixen moviments lents i suaus. Contràriament als entorns de curses d’alta velocitat. Aquesta tesi proposa un nou model d’SLAM, anomenat LIMO-Velo, capaç de corregir el seu estat amb una latència extremadament baixa tractant els punts LiDAR com un flux de dades. Els experiments mostren un salt en robustesa i en la qualitat del mapa mantenint el requisit de correr en temps real. El model aconsegueix una millora relativa del 20% en el KITTI dataset d’odometria respecte al millor rendiment existent; no deriva en un sol esce- nari. La qualitat del mapa a nivell de centı́metre es manté amb velocitats que poden arribar a 20 m/s i 500 graus/s. Utilitzant les biblioteques obertes IKFoM i ikd-Tree, el model funciona x10 més ràpid que la majoria de models d’última generació. Mostrem que LIMO-Velo es pot generalitzar per exe- cutar l’eliminació dinàmica d’objectes, com ara altres agents a la carretera, vianants i altres.Trabajos recientes sobre la localización de vehı́culos y el mapeo de sus en- tornos se desarrollan para dispositivos portátiles o robots terrestres que asumen movimientos lentos y suaves. Al contrario de los entornos de carreras de alta velocidad. Esta tesis propone un nuevo modelo SLAM, LIMO-Velo, capaz de corregir su estado en latencia extremadamente baja al tratar los puntos LiDAR como un flujo de datos. Los experimentos muestran un salto en la solidez y la calidad del mapa mientras se mantiene el requisito de tiempo real. El modelo logra una mejora relativa del 20% en el conjunto de datos de KITTI Odometry sobre el mejor desempeño existente; no deriva en un solo escenario. La calidad del mapa de nivel centimétrico todavı́a se logra a velocidades de carrera que pueden llegar hasta 20 m/s y 500 grados/s. Us- ando las bibliotecas abiertas IKFoM e ikd-Tree, el modelo funciona x10 más rápido que la mayorı́a de los modelos de última generación. Mostramos que LIMO-Velo se puede generalizar para trabajar bajo la eliminación dinámica de objetos, como otros agentes en la carretera, peatones y más.Recent works on localizing vehicles and mapping their environments are de- veloped for handheld devices or terrestrial robots which assume slow and smooth movements. Contrary to high-velocity racing environments. This thesis proposes a new SLAM model, LIMO-Velo, capable of correcting its state at extreme low-latency by treating LiDAR points as a data stream. Experiments show a jump in robustness and map quality while maintaining the real-time requirement. The model achieves a 20% relative improvement on the KITTI Odometry dataset over the existing best performer; it does not drift in a single scenario. Centimeter-level map quality is still achieved under racing velocities that can go up to 20m/s and 500deg/s. Using the IKFoM and ikd-Tree open libraries, the model performs x10 faster than most state-of-the-art models. We show that LIMO-Velo can be generalized to work under dynamic object removal such as other agents in the road, pedestrians, and more.Outgoin