Integration of Absolute Orientation Measurements in the KinectFusion Reconstruction pipeline

In this paper, we show how absolute orientation measurements provided by low-cost but high-fidelity IMU sensors can be integrated into the KinectFusion pipeline. We show that integration improves both runtime, robustness and quality of the 3D reconstruction. In particular, we use this orientation data to seed and regularize the ICP registration technique. We also present a technique to filter the pairs of 3D matched points based on the distribution of their distances. This filter is implemented efficiently on the GPU. Estimating the distribution of the distances helps control the number of iterations necessary for the convergence of the ICP algorithm. Finally, we show experimental results that highlight improvements in robustness, a speed-up of almost 12%, and a gain in tracking quality of 53% for the ATE metric on the Freiburg benchmark.Comment: CVPR Workshop on Visual Odometry and Computer Vision Applications Based on Location Clues 201

Pilot Assisted Inertial Navigation System Aiding Using Bearings-Only Measurements Taken Over Time

The objective of this work is to develop an alternative INS aiding source other than the GPS, while preserving the autonomy of the integrated navigation system. It is proposed to develop a modernized method of aerial navigation using driftmeter measurements from an E/O system for ground feature tracking, and an independent altitude sensor in conjunction with the INS. The pilot will track a ground feature with the E/O system, while the aircraft is on autopilot holding constant airspeed, altitude, and heading during an INS aiding session. The ground feature measurements from the E/O system and the INS output form measurements provided to a linear KF running on the navigation computer to accomplish the INS aiding action. Aiding the INS will be periodically repeated as operationally permissible under pilot discretion. Little to no modeling error will be present when implementing the linear Kalman filter, indicating the strength of the INS aiding action will be exclusively determined by the prevailing degree of observability

On the Integration of Medium Wave Infrared Cameras for Vision-Based Navigation

The ubiquitous nature of GPS has fostered its widespread integration of navigation into a variety of applications, both civilian and military. One alternative to ensure continued flight operations in GPS-denied environments is vision-aided navigation, an approach that combines visual cues from a camera with an inertial measurement unit (IMU) to estimate the navigation states of a moving body. The majority of vision-based navigation research has been conducted in the electro-optical (EO) spectrum, which experiences limited operation in certain environments. The aim of this work is to explore how such approaches extend to infrared imaging sensors. In particular, it examines the ability of medium-wave infrared (MWIR) imagery, which is capable of operating at night and with increased vision through smoke, to expand the breadth of operations that can be supported by vision-aided navigation. The experiments presented here are based on the Minor Area Motion Imagery (MAMI) dataset that recorded GPS data, inertial measurements, EO imagery, and MWIR imagery captured during flights over Wright-Patterson Air Force Base. The approach applied here combines inertial measurements with EO position estimates from the structure from motion (SfM) algorithm. Although precision timing was not available for the MWIR imagery, the EO-based results of the scene demonstrate that trajectory estimates from SfM offer a significant increase in navigation accuracy when combined with inertial data over using an IMU alone. Results also demonstrated that MWIR-based positions solutions provide a similar trajectory reconstruction to EO-based solutions for the same scenes. While the MWIR imagery and the IMU could not be combined directly, through comparison to the combined solution using EO data the conclusion here is that MWIR imagery (with its unique phenomenologies) is capable of expanding the operating envelope of vision-aided navigation

Progress on GPS-Denied, Multi-Vehicle, Fixed-Wing Cooperative Localization

This paper first summarizes recent results of a proposed method for multiple, small, fixed-wing aircraft cooperatively localizing in GPS-denied environments. It then provides a significant future works discussion to provide a vision for the future of cooperative navigation. The goal of this work is to show that many, small, potentially-lower-cost vehicles could collaboratively localize better than a single, more-accurate, higher-cost GPS-denied system. This work is guided by a novel methodology called relative navigation, which has been developed in prior work. Initial work focused on the development and testing of a monocular, visual-inertial odometry for fixed-wing aircraft that accounts for fixed-wing flight characteristics and sensing requirements. The front-end publishes information that enables a back-end where the odometry from multiple vehicles is combined with inter-vehicle measurements and is communicated and shared between vehicles. Each vehicle is able to create a global, backend, graph-based map and optimize it as new information is gained and measurements between vehicles overconstrain the graph. These inter-vehicle measurements allow the optimization to remove accumulated drift for more accurate estimates

로버 항법을 위한 자가보정 영상관성 오도메트리

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2019. 2. 박찬국.This master's thesis presents a direct visual odometry robust to illumination changes and a self-calibrated visual-inertial odometry for a rover localization using an IMU and a stereo camera. Most of the previous vision-based localization algorithms are vulnerable to sudden brightness changes due to strong sunlight or a variance of the exposure time, that violates Lambertian surface assumption. Meanwhile, to decrease the error accumulation of a visual odometry, an IMU can be employed to fill gaps between successive images. However, extrinsic parameters for a visual-inertial system should be computed precisely since they play an important role in making a bridge between the visual and inertial coordinate frames, spatially as well as temporally. This thesis proposes a bucketed illumination model to account for partial and global illumination changes along with a framework of a direct visual odometry for a rover localization. Furthermore, this study presents a self-calibrated visual-inertial odometry in which the time-offset and relative pose of an IMU and a stereo camera are estimated by using point feature measurements. Specifically, based on the extended Kalman filter pose estimator, the calibration parameters are augmented in the filter state. The proposed visual odometry is evaluated through the open source dataset where images are captured in a Lunar-like environment. In addition to this, we design a rover using commercially available sensors, and a field testing of the rover confirms that the self-calibrated visual-inertial odometry decreases a localization error in terms of a return position by 76.4% when compared to the visual-inertial odometry without the self-calibration.본 논문에서는 로버 항법 시스템을 위해 관성측정장치와 스테레오 카메라를 사용하여 빛 변화에 강건한 직접 방식 영상 오도메트리와 자가 보정 영상관성 항법 알고리즘을 제안한다. 기존 대부분의 영상기반 항법 알고리즘들은 램버션 표면 가정을 위배하는 야외의 강한 햇빛 혹은 일정하지 않은 카메라의 노출 시간으로 인해 영상의 밝기 변화에 취약하였다. 한편, 영상 오도메트리의 오차 누적을 줄이기 위해 관성측정장치를 사용할 수 있지만, 영상관성 시스템에 대한 외부 교정 변수는 공간 및 시간적으로 영상 및 관성 좌표계를 연결하기 때문에 사전에 정확하게 계산되어야 한다. 본 논문은 로버 항법을 위해 지역 및 전역적인 빛 변화를 설명하는 직접 방식 영상 오도메트리의 버킷 밝기 모델을 제안한다. 또한, 본 연구에서는 스트레오 카메라에서 측정된 특징점을 이용하여 관성측정장치와 카메라간의 시간 오프셋과 상대 위치 및 자세를 추정하는 자가 보정 영상관성 항법 알고리즘을 제시한다. 특히, 제안하는 영상관성 알고리즘은 확장 칼만 필터에 기반하며 교정 파라미터를 필터의 상태변수에 확장하였다. 제안한 직접방식 영상 오도메트리는 달 유사환경에서 촬영된 오픈소스 데이터셋을 통해 그 성능을 검증하였다. 또한 상용 센서 및 로버 플랫폼을 이용하여 테스트 로버를 설계하였고, 이를 통해 영상관성 시스템을 자가 보정 할 경우 그렇지 않은 경우 보다 회기 위치 오차(return position error)가 76.4% 감소됨을 확인하였다.Abstract Contents List of Tables List of Figures Chapter 1 Introduction 1.1 Motivation and background 1.2 Objectives and contributions Chapter 2 Related Works 2.1 Visual odometry 2.2 Visual-inertial odometry Chapter 3 Direct Visual Odometry at Outdoor 3.1 Direct visual odometry 3.1.1 Notations 3.1.2 Camera projection model 3.1.3 Photometric error 3.2 The proposed algorithm 3.2.1 Problem formulation 3.2.2 Bucketed illumination model 3.2.3 Adaptive prior weight 3.3 Experimental results 3.3.1 Synthetic image sequences 3.3.2 MAV datasets 3.3.3 Planetary rover datasets Chapter 4 Self-Calibrated Visual-Inertial Odometry 4.1 State representation 4.1.1 IMU state 4.1.2 Calibration parameter state 4.2 State-propagation 4.3 Measurement-update 4.3.1 Point feature measurement 4.3.2 Measurement error modeling 4.4 Experimental results 4.4.1 Hardware setup 4.4.2 Vision front-end design 4.4.3 Rover field testing Chapter 5 Conclusions 5.1 Conclusion and summary 5.2 Future works Bibliography Chapter A Derivation of Photometric Error Jacobian 국문 초록Maste