Keyframe-based visual–inertial odometry using nonlinear optimization

Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate visual–inertial odometry or simultaneous localization and mapping (SLAM). While historically the problem has been addressed with filtering, advancements in visual estimation suggest that nonlinear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual–inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter. This competitive reference implementation performs tightly coupled filtering-based visual–inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy

KEYFRAME-BASED VISUAL-INERTIAL SLAM USING NONLINEAR OPTIMIZATION

Abstract—The fusion of visual and inertial cues has become popular in robotics due to the complementary nature of the two sensing modalities. While most fusion strategies to date rely on filtering schemes, the visual robotics community has recently turned to non-linear optimization approaches for tasks such as visual Simultaneous Localization And Mapping (SLAM), following the discovery that this comes with significant advantages in quality of performance and computational complexity. Following this trend, we present a novel approach to tightly integrate visual measurements with readings from an Inertial Measurement Unit (IMU) in SLAM. An IMU error term is integrated with the landmark reprojection error in a fully probabilistic manner, resulting to a joint non-linear cost function to be optimized. Employing the powerful concept of ‘keyframes ’ we partially marginalize old states to maintain a bounded-sized optimization window, ensuring real-time operation. Comparing against both vision-only and loosely-coupled visual-inertial algorithms, our experiments confirm the benefits of tight fusion in terms of accuracy and robustness. I

A factorization approach to inertial affine structure from motion

We consider the problem of reconstructing a 3-D scene from a moving camera with high frame rate using the affine projection model. This problem is traditionally known as Affine Structure from Motion (Affine SfM), and can be solved using an elegant low-rank factorization formulation. In this paper, we assume that an accelerometer and gyro are rigidly mounted with the camera, so that synchronized linear acceleration and angular velocity measurements are available together with the image measurements. We extend the standard Affine SfM algorithm to integrate these measurements through the use of image derivatives

A factorization approach to inertial affine structure from motion

We consider the problem of reconstructing a 3-D scene from a moving camera with high frame rate using the affine projection model. This problem is traditionally known as Affine Structure from Motion (Affine SfM), and can be solved using an elegant low-rank factorization formulation. In this paper, we assume that an accelerometer and gyro are rigidly mounted with the camera, so that synchronized linear acceleration and angular velocity measurements are available together with the image measurements. We extend the standard Affine SfM algorithm to integrate these measurements through the use of image derivatives

Visual 3-D SLAM from UAVs

The aim of the paper is to present, test and discuss the implementation of Visual SLAM techniques to images taken from Unmanned Aerial Vehicles (UAVs) outdoors, in partially structured environments. Every issue of the whole process is discussed in order to obtain more accurate localization and mapping from UAVs flights. Firstly, the issues related to the visual features of objects in the scene, their distance to the UAV, and the related image acquisition system and their calibration are evaluated for improving the whole process. Other important, considered issues are related to the image processing techniques, such as interest point detection, the matching procedure and the scaling factor. The whole system has been tested using the COLIBRI mini UAV in partially structured environments. The results that have been obtained for localization, tested against the GPS information of the flights, show that Visual SLAM delivers reliable localization and mapping that makes it suitable for some outdoors applications when flying UAVs

Galileo and EGNOS as an asset for UTM safety and security

GAUSS (Galileo-EGNOS as an Asset for UTM Safety and Security) is a H2020 project1 that aims at designing and developing high performance positioning systems for drones within the U-Space framework focusing on UAS (Unmanned Aircraft System) VLL (Very Low Level) operations. The key element within GAUSS is the integration and exploitation of Galileo and EGNOS exceptional features in terms of accuracy, integrity and security, which will be key assets for the safety of current and future drone operations. More concretely, high accuracy, authentication, precise timing (among others) are key GNSS (Global Navigation Satellite System) enablers of future integrated drone operations under UTM (UAS Traffic Management) operations, which in Europe will be deployed under U-Space [1]. The U-Space concept helps control, manage and integrate all UAS in the VLL airspace to ensure the security and efficiency of UAS operations. GAUSS will enable not only safe, timely and efficient operations but also coordination among a higher number of RPAS (Remotely Piloted Aircraft System) in the air with the appropriate levels of security, as it will improve anti-jamming and anti-spoofing capabilities through a multi-frequency and multi-constellation approach and Galileo authentication operations. The GAUSS system will be validated with two field trials in two different UTM real scenarios (in-land and sea) with the operation of a minimum of four UTM coordinated UAS from different types (fixed and rotary wing), manoeuvrability and EASA (European Aviation Safety Agency) operational categories. The outcome of the project will consist of Galileo-EGNOS based technological solutions to enhance safety and security levels in both, current UAS and future UTM operations. Increased levels of efficiency, reliability, safety, and security in UAS operations are key enabling features to foster the EU UAS regulation, market development and full acceptance by the society.Peer ReviewedPostprint (author's final draft

Improving Real-World Performance of Vision Aided Navigation in a Flight Environment

The motivation of this research is to fuse information from an airborne imaging sensor with information extracted from satellite imagery in order to provide accurate position when GPS is unavailable for an extended duration. A corpus of existing geo-referenced satellite imagery is used to create a key point database. A novel algorithm for recovering coarse pose using by comparing key points extracted from the airborne imagery to the reference database is developed. This coarse position is used to bootstrap a local-area geo-registration algorithm, which provides GPS-level position estimates. This research derives optimizations for existing local-area methods for operation in flight environments

Cooperative monocular-based SLAM for multi-UAV systems in GPS-denied environments

This work presents a cooperative monocular-based SLAM approach for multi-UAV systems that can operate in GPS-denied environments. The main contribution of the work is to show that, using visual information obtained from monocular cameras mounted onboard aerial vehicles flying in formation, the observability properties of the whole system are improved. This fact is especially notorious when compared with other related visual SLAM configurations. In order to improve the observability properties, some measurements of the relative distance between the UAVs are included in the system. These relative distances are also obtained from visual information. The proposed approach is theoretically validated by means of a nonlinear observability analysis. Furthermore, an extensive set of computer simulations is presented in order to validate the proposed approach. The numerical simulation results show that the proposed system is able to provide a good position and orientation estimation of the aerial vehicles flying in formation.Peer ReviewedPostprint (published version

MagSLAM: Aerial Simultaneous Localization and Mapping using Earth\u27s Magnetic Anomaly Field

Instances of spoofing and jamming of global navigation satellite systems (GNSSs) have emphasized the need for alternative navigation methods. Aerial navigation by magnetic map matching has been demonstrated as a viable GNSS‐alternative navigation technique. Flight test demonstrations have achieved accuracies of tens of meters over hour‐long flights, but these flights required accurate magnetic maps which are not always available. Magnetic map availability and resolution vary widely around the globe. Removing the dependency on prior survey maps extends the benefits of aerial magnetic navigation methods to small unmanned aerial systems (sUAS) at lower altitudes where magnetic maps are especially undersampled or unavailable. In this paper, a simultaneous localization and mapping (SLAM) algorithm known as FastSLAM was modified to use scalar magnetic measurements to constrain a drifting inertial navigation system (INS). The algorithm was then demonstrated on real magnetic navigation flight test data. Similar in performance to the map‐based approach, MagSLAM achieved tens of meters accuracy in a 100‐minute flight without the use of a prior magnetic map. Aerial SLAM using Earth\u27s magnetic anomaly field provides a GNSS‐alternative navigation method that is globally persistent, impervious to jamming or spoofing, stealthy, and locally accurate to tens of meters without the need for a magnetic map