Large-scale monocular SLAM by local bundle adjustment and map joining

This paper first demonstrates an interesting property of bundle adjustment (BA), "scale drift correction". Here "scale drift correction" means that BA can converge to the correct solution (up to a scale) even if the initial values of the camera pose translations and point feature positions are calculated using very different scale factors. This property together with other properties of BA makes it the best approach for monocular Simultaneous Localization and Mapping (SLAM), without considering the computational complexity. This naturally leads to the idea of using local BA and map joining to solve large-scale monocular SLAM problem, which is proposed in this paper. The local maps are built through Scale-Invariant Feature Transform (SIFT) for feature detection and matching, random sample consensus (RANSAC) paradigm at different levels for robust outlier removal, and BA for optimization. To reduce the computational cost of the large-scale map building, the features in each local map are judiciously selected and then the local maps are combined using a recently developed 3D map joining algorithm. The proposed large-scale monocular SLAM algorithm is evaluated using a publicly available dataset with centimeter-level ground truth. ©2010 IEEE

Fusion of Visible and Thermal-Infrared Imagery for SLAM for Landing on Icy Moons

This paper addresses the problem of localization for landing on the surface of icy moons, like Europa or Enceladus. Due to the possibility of specular reflection as well as high bulk albedo, icy surfaces present new challenges that make traditional vision-based navigation systems relying on visible imagery unreliable. We propose augmenting visible light cameras with a thermal-infrared camera using inverse-depth parameterized monocular EKF-SLAM to address problems arising from the appearance of icy moons. Results were obtained from a novel procedural Europa surface simulation which models the appearance and the thermal properties simultaneously from physically-based methods. In this framework, we show that thermal features improve localization by 23% on average when compared to a visible camera. Moreover, fusing both sensing modalities increases the improvement in localization to 31% on average, compared to using a visible light camera alone

Collaborative Monocular SLAM with Multiple Micro Aerial Vehicles

This paper presents a framework for collaborative localization and mapping with multiple Micro Aerial Vehicles (MAVs) in unknown environments. Each MAV estimates its motion individually using an onboard, monocular visual odometry algorithm. The system of MAVs acts as a distributed preprocessor that streams only features of selected keyframes and relative-pose estimates to a centralized ground station. The ground station creates an individual map for each MAV and merges them together whenever it detects overlaps. This allows the MAVs to express their position in a common, global coordinate frame. The key to real-time performance is the design of data-structures and processes that allow multiple threads to concurrently read and modify the same map. The presented framework is tested in both indoor and outdoor environments with up to three MAVs. To the best of our knowledge, this is the first work on real-time collaborative monocular SLAM, which has also been applied to MAVs

A collaborative monocular visual simultaneous localization and mapping solution to generate a semi-dense 3D map.

The utilization and generation of indoor maps are critical in accurate indoor tracking. Simultaneous Localization and Mapping (SLAM) is one of the main techniques used for such map generation. In SLAM, an agent generates a map of an unknown environment while approximating its own location in it. The prevalence and afford-ability of cameras encourage the use of Monocular Visual SLAM, where a camera is the only sensing device for the SLAM process. In modern applications, multiple mobile agents may be involved in the generation of indoor maps, thus requiring a distributed computational framework. Each agent generates its own local map, which can then be combined with those of other agents into a map covering a larger area. In doing so, they cover a given environment faster than a single agent. Furthermore, they can interact with each other in the same environment, making this framework more practical, especially for collaborative applications such as augmented reality. One of the main challenges of collaborative SLAM is identifying overlapping maps, especially when the relative starting positions of the agents are unknown. We propose a system comprised of multiple monocular agents with unknown relative starting positions to generate a semi-dense global map of the environment

Mobile Robot Manipulator System Design for Localization and Mapping in Cluttered Environments

In this thesis, a compact mobile robot has been developed to build real-time 3D maps of hazards and cluttered environments inside damaged buildings for rescue tasks using visual Simultaneous Localization And Mapping (SLAM) algorithms. In order to maximize the survey area in such environments, this mobile robot is designed with four omni-wheels and equipped with a 6 Degree of Freedom (DOF) robotic arm carrying a stereo camera mounted on its end-effector. The aim of using this mobile articulated robotic system is monitor different types of regions within the area of interest, ranging from wide open spaces to smaller and irregular regions behind narrow gaps. In the first part of the thesis, the robot system design is presented in detail, including the kinematic systems of the omni-wheeled mobile platform and the 6-DOF robotic arm, estimation of the biases in parameters of these kinematic systems, the sensors and calibration of their parameters. These parameters are important for the sensor fusion utilized in the next part of the thesis, where two operation modes are proposed to retain the camera pose when the visual SLAM algorithms fail due to variety of the region types. In the second part, an integrated sensor data fusion, odometry and SLAM scheme is developed, where the camera poses are estimated using forward kinematic equations of the robotic arm and fused to the visual SLAM and odometry algorithms. A modified wavefront algorithm with reduced computational complexity is used to find the shortest path to reach the identified goal points. Finally, a dynamic control scheme is developed for path tracking and motion control of the mobile platform and the robot arm, with sub-systems in the form of PD controllers and extended Kalman filters. The overall system design is physically implemented on a prototype integrated mobile robot platform and successfully tested in real-time

Fusing Monocular Information in Multicamera SLAM

International audienceThis paper explores the possibilities of using monocu-lar simultaneous localization and mapping (SLAM) algorithms in systems with more than one camera. The idea is to combine in a single system the advantages of both monocular vision (bearings-only, infinite range observations but no 3-D instantaneous information) and stereovision (3-D information up to a limited range). Such a system should be able to instantaneously map nearby objects while still considering the bearing information provided by the observation of remote ones. We do this by considering each camera as an independent sensor rather than the entire set as a monolithic su-persensor. The visual data are treated by monocular methods and fused by the SLAM filter. Several advantages naturally arise as interesting possibilities, such as the desynchronization of the firing of the sensors, the use of several unequal cameras, self-calibration, and cooperative SLAM with several independently moving cameras. We validate the approach with two different applications: a stereovision SLAM system with automatic self-calibration of the rig's main extrinsic parameters and a cooperative SLAM system with two independent free-moving cameras in an outdoor setting

Distributed consensus in multi-robot systems with visual perception

La idea de equipos de robots actuando con autonomía y de manera cooperativa está cada día más cerca de convertirse en realidad. Los sistemas multi robot pueden ejecutar tareas de gran complejidad con mayor robustez y en menos tiempo que un robot trabajando solo. Por otra parte, la coordinación de un equipo de robots introduce complicaciones que los ingenieros encargados de diseñar estos sistemas deben afrontar. Conseguir que la percepción del entorno sea consistente en todos los robots es uno de los aspectos más importantes requeridos en cualquier tarea cooperativa, lo que implica que las observaciones de cada robot del equipo deben ser transmitidas a todos los otros miembros. Cuando dos o más robots poseen información común del entorno, el equipo debe alcanzar un consenso usando toda la información disponible. Esto se debe hacer considerando las limitaciones de cada robot, teniendo en cuenta que no todos los robots se pueden comunicar unos con otros. Con este objetivo, se aborda la tarea de diseñar algoritmos distribuidos que consigan que un equipo de robots llegue a un consenso acerca de la información percibida por todos los miembros. Específicamente, nos centramos en resolver este problema cuando los robots usan la visión como sensor para percibir el entorno. Las cámaras convencionales son muy útiles a la hora de ejecutar tareas como la navegación y la construcción de mapas, esenciales en el ámbito de la robótica, gracias a la gran cantidad de información que contiene cada imagen. Sin embargo, el uso de estos sensores en un marco distribuido introduce una gran cantidad de complicaciones adicionales que deben ser abordadas si se quiere cumplir el objetivo propuesto. En esta Tesis presentamos un estudio profundo de los algoritmos distribuidos de consenso y cómo estos pueden ser usados por un equipo de robots equipados con cámaras convencionales, resolviendo los aspectos más importantes relacionados con el uso de estos sensores. En la primera parte de la Tesis nos centramos en encontrar correspondencias globales entre las observaciones de todos los robots. De esta manera, los robots son capaces de detectar que observaciones deben ser combinadas para el cálculo del consenso. También lidiamos con el problema de la robustez y la detección distribuida de espurios durante el cálculo del consenso. Para contrarrestar el incremento del tamaño de los mensajes intercambiados por los robots en las etapas anteriores, usamos las propiedades de los polinomios de Chebyshev, reduciendo el número de iteraciones que se requieren para alcanzar el consenso. En la segunda parte de la Tesis, centramos nuestra atención en los problemas de crear un mapa y controlar el movimiento del equipo de robots. Presentamos soluciones para alcanzar un consenso en estos escenarios mediante el uso de técnicas de visión por computador ampliamente conocidas. El uso de algoritmos de estructura y movimiento nos permite obviar restricciones tales como que los robots tengan que observarse unos a otros directamente durante el control o la necesidad de especificar un marco de referencia común. Adicionalmente, nuestros algoritmos tienen un comportamiento robusto cuando la calibración de las cámaras no se conoce. Finalmente, la evaluación de las propuestas se realiza utilizando un data set de un entorno urbano y robots reales con restricciones de movimiento no holónomas. Todos los algoritmos que se presentan en esta Tesis han sido diseñados para ser ejecutados de manera distribuida. En la Tesis demostramos de manera teórica las principales propiedades de los algoritmos que se proponen y evaluamos la calidad de los mismos con datos simulados e imágenes reales. En resumen, las principales contribuciones de esta Tesis son: • Un conjunto de algoritmos distribuidos que permiten a un equipo de robots equipados con cámaras convencionales alcanzar un consenso acerca de la información que perciben. En particular, proponemos tres algoritmos distribuidos con el objetivo de resolver los problemas de encontrar correspondencias globales entre la información de todos los robots, detectar y descartar información espuria, y reducir el número de veces que los robots tienen que comunicarse entre ellos antes de alcanzar el consenso. • La combinación de técnicas de consenso distribuido y estructura y movimiento en tareas de control y percepción. Se ha diseñado un algoritmo para construir un mapa topológico de manera cooperativa usando planos como características del mapa y restricciones de homografía como elementos para relacionar las observaciones de los robots. También se ha propuesto una ley de control distribuida utilizando la geometría epipolar con el objetivo de hacer que el equipo de robots alcance una orientación común sin la necesidad de observarse directamente unos a otros

Recovering Scale in Relative Pose and Target Model Estimation Using Monocular Vision

A combined relative pose and target object model estimation framework using a monocular camera as the primary feedback sensor has been designed and validated in a simulated robotic environment. The monocular camera is mounted on the end-effector of a robot manipulator and measures the image plane coordinates of a set of point features on a target workpiece object. Using this information, the relative position and orientation, as well as the geometry, of the target object are recovered recursively by a Kalman filter process. The Kalman filter facilitates the fusion of supplemental measurements from range sensors, with those gathered with the camera. This process allows the estimated system state to be accurate and recover the proper environment scale. Current approaches in the research areas of visual servoing control and mobile robotics are studied in the case where the target object feature point geometry is well-known prior to the beginning of the estimation. In this case, only the relative pose of target object frames is estimated over a sequence of frames from a single monocular camera. An observability analysis was carried out to identify the physical configurations of camera and target object for which the relative pose cannot be recovered by measuring only the camera image plane coordinates of the object point features. A popular extension to this is to concurrently estimate the target object model concurrently with the relative pose of the camera frame, a process known as Simultaneous Localization and Mapping (SLAM). The recursive framework was augmented to facilitate this larger estimation problem. The scale of the recovered solution is ambiguous using measurements from a single camera. A second observability analysis highlights more configurations for which the relative pose and target object model are unrecoverable from camera measurements alone. Instead, measurements which contain the global scale are required to obtain an accurate solution. A set of additional sensors are detailed, including range finders and additional cameras. Measurement models for each are given, which facilitate the fusion of this supplemental data with the original monocular camera image measurements. A complete framework is then derived to combine a set of such sensor measurements to recover an accurate relative pose and target object model estimate. This proposed framework is tested in a simulation environment with a virtual robot manipulator tracking a target object workpiece through a relative trajectory. All of the detailed estimation schemes are executed: the single monocular camera cases when the target object geometry are known and unknown, respectively; a two camera system in which the measurements are fused within the Kalman filter to recover the scale of the environment; a camera and point range sensor combination which provides a single range measurement at each system time step; and a laser pointer and camera hybrid which concurrently tries to measure the feature point images and a single range metric. The performance of the individual test cases are compared to determine which set of sensors is able to provide robust and reliable estimates for use in real world robotic applications. Finally, some conclusions on the performance of the estimators are drawn and directions for future work are suggested. The camera and range finder combination is shown to accurately recover the proper scale for the estimate and warrants further investigation. Further, early results from the multiple monocular camera setup show superior performance to the other sensor combinations and interesting possibilities are available for wide field-of-view super sensors with high frame rates, built from many inexpensive devices