A hybrid visual-based SLAM architecture: local filter-based SLAM with keyframe-based global mapping

This work presents a hybrid visual-based SLAM architecture that aims to take advantage of the strengths of each of the two main methodologies currently available for implementing visual-based SLAM systems, while at the same time minimizing some of their drawbacks. The main idea is to implement a local SLAM process using a filter-based technique, and enable the tasks of building and maintaining a consistent global map of the environment, including the loop closure problem, to use the processes implemented using optimization-based techniques. Different variants of visual-based SLAM systems can be implemented using the proposed architecture. This work also presents the implementation case of a full monocular-based SLAM system for unmanned aerial vehicles that integrates additional sensory inputs. Experiments using real data obtained from the sensors of a quadrotor are presented to validate the feasibility of the proposed approachPostprint (published version

RGB-D Odometry and SLAM

The emergence of modern RGB-D sensors had a significant impact in many application fields, including robotics, augmented reality (AR) and 3D scanning. They are low-cost, low-power and low-size alternatives to traditional range sensors such as LiDAR. Moreover, unlike RGB cameras, RGB-D sensors provide the additional depth information that removes the need of frame-by-frame triangulation for 3D scene reconstruction. These merits have made them very popular in mobile robotics and AR, where it is of great interest to estimate ego-motion and 3D scene structure. Such spatial understanding can enable robots to navigate autonomously without collisions and allow users to insert virtual entities consistent with the image stream. In this chapter, we review common formulations of odometry and Simultaneous Localization and Mapping (known by its acronym SLAM) using RGB-D stream input. The two topics are closely related, as the former aims to track the incremental camera motion with respect to a local map of the scene, and the latter to jointly estimate the camera trajectory and the global map with consistency. In both cases, the standard approaches minimize a cost function using nonlinear optimization techniques. This chapter consists of three main parts: In the first part, we introduce the basic concept of odometry and SLAM and motivate the use of RGB-D sensors. We also give mathematical preliminaries relevant to most odometry and SLAM algorithms. In the second part, we detail the three main components of SLAM systems: camera pose tracking, scene mapping and loop closing. For each component, we describe different approaches proposed in the literature. In the final part, we provide a brief discussion on advanced research topics with the references to the state-of-the-art.Comment: This is the pre-submission version of the manuscript that was later edited and published as a chapter in RGB-D Image Analysis and Processin

Online Synthesis Of Speculative Building Information Models For Robot Motion Planning

Autonomous mobile robots today still lack the necessary understanding of indoor environments for making informed decisions about the state of the world beyond their immediate field of view. As a result, they are forced to make conservative and often inaccurate assumptions about unexplored space, inhibiting the degree of performance being increasingly expected of them in the areas of high-speed navigation and mission planning. In order to address this limitation, this thesis explores the use of Building Information Models (BIMs) for providing the existing ecosystem of local and global planning algorithms with informative compact higher-level representations of indoor environments. Although BIMs have long been used in architecture, engineering, and construction for a number of different purposes, to our knowledge, this is the first instance of them being used in robotics. Given the technical constraints accompanying this domain, including a limited and incomplete set of observations which grows over time, the systems we present are designed such that together they produce BIMs capable of providing explanations of both the explored and unexplored space in an online fashion. The first is a SLAM system that uses the structural regularity of buildings in order to mitigate drift and provide the simplest explanation of architectural features such as floors, walls, and ceilings. The planar model generated is then passed to a secondary system that then reasons about their mutual relationships in order to provide a water-tight model of the observed and inferred freespace. Our experimental results demonstrate this to be an accurate and efficient approach towards this end

실내 로봇을 위한 영상 기반 주행 거리 기록계

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2019. 2. 김현진.본 논문에서는 카메라로부터 촬영되는 일련의 연속적인 이미지들로부터, 3차원 공간상에서 자기 자신의 6 자유도 움직임을 추정하는 기법인 영상 기반 주행 거리 기록계 (VO) 그리고 동시적 위치 인식 및 지도 작성 (vSLAM) 기법들에 대해 탐구하였고, 특히 주변 환경 조건에 대해 강건하고, 정확한 위치 추정 기법들을 새롭게 제안하였다. 영상 내 갑작스럽게 발생하는 빛 변화를 간단한 아핀 변화 모델로 모사함으로써 조도 변화에도 강건한 직접적 방식 기반의 위치 추정 알고리즘을 새롭게 제안하였다. 또한 선, 면과 같은 실내 구조적 특징들을 효과적으로 동시에 이용함으로써 매우 정확한 위치 인식 기법을 새롭게 연구하였다. 이렇게 새롭게 제안한 기법들은 갑작스러운 조도 변화가 일어나는 실내 및 실외, 그리고 카메라의 순수한 제자리 회전 운동과 같은 도전적인 환경과 움직임에서도 정확도를 잃지 않는다는 것이 가장 중요한 특징이다. 첫 번째로, 영상 내 조도 변화에 대해 강건하게 대응하는 직접적 방식 기반의 영상 기반 주행 거리 기록계 알고리즘을 새롭게 제안하였다. 일반적인 직접적 방식 기반의 알고리즘은 영상 내 동일 물체는 동일 밝기를 가진다는 밝기 불변량 가정을 기반으로 위치 추정을 수행하기 때문에 조그마한 빛 변화에도 매우 취약하고 불안정한 성능을 보여준다. 제안한 직접적 방식 기반의 알고리즘은 영상 내 여러 개의 패치를 생성한 뒤, 각 패치별로 독립적인 아핀 빛 변화 모델을 적용하고 이를 실시간으로 추정, 보상함으로써 전역적 및 부분적 조도 변화에 매우 강건한 성능을 보여준다. 추가적으로 특징점 기반 알고리즘을 사전 움직임으로 결합하고 이를 위치 추정 최적화 시에 사용하여, 기존 알고리즘 대비 더 높은 정확도와 더 안정적인 자가 위치 추정 성능을 얻을 수 있었다. 다양하고 도전적인 환경에서 촬영된 영상 데이터셋을 이용하여, 타 알고리즘 대비 제안한 방법의 효율성 및 위치 추정 성능을 정량적으로 평가해보았으며, 추가적으로 실내 비행 드론에 탑재하여 도전적인 조도 변화 환경 내에서도 영상 기반 자율 비행이 가능함을 제시하였다. 두 번째로는, 컬러 및 깊이 영상에서 발견되는 선 및 면과 같은 공간 구조적 특징을 적극적으로 활용한 고정밀 영상 기반 주행 기록계 알고리즘을 새롭게 연구하였다. 대부분의 영상 기반 위치 추정 알고리즘들은 시간이 지남이 따라 누적되는 큰 위치 추정 오차를 피할 수 없는데, 이는 대부분 부정확한 회전 운동 추정으로 인해 유발된다고 알려져 있다. 새롭게 제안한 알고리즘은 실내 구조적 특징인 선과 면을 추적하여, 이러한 위치 추정 오차의 주요 원인인 회전운동을 매우 정확하고 드리프트 없이 추정할 수 있었다. 우리는 효율적인 SO(3) 공간상에서 제한된 평균 이동 알고리즘을 사용하여 환경의 구조적 규칙성을 인지하고 추적하였다. 이렇게 정확한 카메라의 회전 움직임이 얻어진 후, 이를 이용하여 회전 성분이 제거된 재투영 오차를 최소화하여 카메라의 병진 움직임을 추정한다. 제안된 위치 추정 알고리즘은 순수 회전 움직임과 같이 추정하기 어려운 카메라 움직임이 포함된 다양한 영상 데이터셋에서 평가되었으며, 특히 타 알고리즘 대비 매우 높은 정확성과 강건성, 그리고 낮은 드리프트 오차를 실험적으로 보여주었다.This thesis explores the robust and accurate 6-DoF camera motion estimation from a sequence of images, called visual odometry (VO) or visual simultaneous localization and mapping (vSLAM). We focus on the robustness and high accuracy of the VO and visual localization by explicitly modeling the light changes as an affine illumination model, and utilizing the indoor environmental structures such as lines and planes. This brings the significant advantage to VO that it does not lose estimation accuracy in challenging environments such as light-changing conditions or pure, on the spot rotations. The first part of the thesis proposes a novel patch-based illumination invariant visual odometry algorithm (PIVO). PIVO employs an affine illumination change model per each patch in the image to compensate unexpected, abrupt, and irregular illumination changes during the direct motion estimation. PIVO infers camera geometry directly from the images, i.e., the raw sensor measurements, without intermediate abstraction, for instance in the form of keypoint matches. We furthermore incorporate a motion prior from feature-based stereo visual odometry in the optimization, resulting in higher accuracy and more stable motion estimates. We evaluate the proposed VO algorithm on a variety of datasets, and demonstrate autonomous flight experiments with an aerial robot, showing that the proposed method successfully estimates 6-DoF pose under significant illumination changes. In the second part of the thesis, we propose a low-drift VO that separately estimates rotational and translational motion from lines, planes, and points found in RGB-D images. To estimate the rotational motion that is a main source of drift in VO in an accurate and drift-free manner, we exploit both lines and planes jointly from environmental regularities. We recognize and track the structural regularities with an efficient SO(3)-manifold constrained mean shift algorithm. Once the absolute camera orientation is found, we recover the translational motion from all tracked points with and without depth by minimizing the de-rotated reprojection error. We compare the proposed algorithm to other state-of-the-art VO methods on a variety of RGB-D datasets that include especially challenging pure rotations, and demonstrate improved accuracy and lower drift error.1 Introduction 2 Background 3 Robust Visual Odometry to Irregular Illumination Changes with RGB-D Camera 4 Robust Visual Localization in Changing Lighting Conditions 5 Autonomous Flight with Robust Visual Odometry under Dynamic Lighting Conditions 6 Visual Odometry with Drift-Free Rotation Estimation Using Indoor Scene Regularities 7 Low-Drift Visual Odometry in Structured Environments by Decoupling Rotational and Translational Motion 8 Indoor RGB-D Compass from a Single Line and Plane 9 Linear RGB-D SLAM for Planar Environments 10 ConclusionDocto