Dynamic Body VSLAM with Semantic Constraints

Image based reconstruction of urban environments is a challenging problem that deals with optimization of large number of variables, and has several sources of errors like the presence of dynamic objects. Since most large scale approaches make the assumption of observing static scenes, dynamic objects are relegated to the noise modeling section of such systems. This is an approach of convenience since the RANSAC based framework used to compute most multiview geometric quantities for static scenes naturally confine dynamic objects to the class of outlier measurements. However, reconstructing dynamic objects along with the static environment helps us get a complete picture of an urban environment. Such understanding can then be used for important robotic tasks like path planning for autonomous navigation, obstacle tracking and avoidance, and other areas. In this paper, we propose a system for robust SLAM that works in both static and dynamic environments. To overcome the challenge of dynamic objects in the scene, we propose a new model to incorporate semantic constraints into the reconstruction algorithm. While some of these constraints are based on multi-layered dense CRFs trained over appearance as well as motion cues, other proposed constraints can be expressed as additional terms in the bundle adjustment optimization process that does iterative refinement of 3D structure and camera / object motion trajectories. We show results on the challenging KITTI urban dataset for accuracy of motion segmentation and reconstruction of the trajectory and shape of moving objects relative to ground truth. We are able to show average relative error reduction by a significant amount for moving object trajectory reconstruction relative to state-of-the-art methods like VISO 2, as well as standard bundle adjustment algorithms

Lifting GIS Maps into Strong Geometric Context for Scene Understanding

Contextual information can have a substantial impact on the performance of visual tasks such as semantic segmentation, object detection, and geometric estimation. Data stored in Geographic Information Systems (GIS) offers a rich source of contextual information that has been largely untapped by computer vision. We propose to leverage such information for scene understanding by combining GIS resources with large sets of unorganized photographs using Structure from Motion (SfM) techniques. We present a pipeline to quickly generate strong 3D geometric priors from 2D GIS data using SfM models aligned with minimal user input. Given an image resectioned against this model, we generate robust predictions of depth, surface normals, and semantic labels. We show that the precision of the predicted geometry is substantially more accurate other single-image depth estimation methods. We then demonstrate the utility of these contextual constraints for re-scoring pedestrian detections, and use these GIS contextual features alongside object detection score maps to improve a CRF-based semantic segmentation framework, boosting accuracy over baseline models

Continuous Modeling of 3D Building Rooftops From Airborne LIDAR and Imagery

In recent years, a number of mega-cities have provided 3D photorealistic virtual models to support the decisions making process for maintaining the cities' infrastructure and environment more effectively. 3D virtual city models are static snap-shots of the environment and represent the status quo at the time of their data acquisition. However, cities are dynamic system that continuously change over time. Accordingly, their virtual representation need to be regularly updated in a timely manner to allow for accurate analysis and simulated results that decisions are based upon. The concept of "continuous city modeling" is to progressively reconstruct city models by accommodating their changes recognized in spatio-temporal domain, while preserving unchanged structures. However, developing a universal intelligent machine enabling continuous modeling still remains a challenging task. Therefore, this thesis proposes a novel research framework for continuously reconstructing 3D building rooftops using multi-sensor data. For achieving this goal, we first proposes a 3D building rooftop modeling method using airborne LiDAR data. The main focus is on the implementation of an implicit regularization method which impose a data-driven building regularity to noisy boundaries of roof planes for reconstructing 3D building rooftop models. The implicit regularization process is implemented in the framework of Minimum Description Length (MDL) combined with Hypothesize and Test (HAT). Secondly, we propose a context-based geometric hashing method to align newly acquired image data with existing building models. The novelty is the use of context features to achieve robust and accurate matching results. Thirdly, the existing building models are refined by newly proposed sequential fusion method. The main advantage of the proposed method is its ability to progressively refine modeling errors frequently observed in LiDAR-driven building models. The refinement process is conducted in the framework of MDL combined with HAT. Markov Chain Monte Carlo (MDMC) coupled with Simulated Annealing (SA) is employed to perform a global optimization. The results demonstrates that the proposed continuous rooftop modeling methods show a promising aspects to support various critical decisions by not only reconstructing 3D rooftop models accurately, but also by updating the models using multi-sensor data

3D Reconstruction of Building Rooftop and Power Line Models in Right-of-Ways Using Airborne LiDAR Data

The research objectives aimed to achieve thorough the thesis are to develop methods for reconstructing models of building and PL objects of interest in the power line (PL) corridor area from airborne LiDAR data. For this, it is mainly concerned with the model selection problem for which model is more optimal in representing the given data set. This means that the parametric relations and geometry of object shapes are unknowns and optimally determined by the verification of hypothetical models. Therefore, the proposed method achieves high adaptability to the complex geometric forms of building and PL objects. For the building modeling, the method of implicit geometric regularization is proposed to rectify noisy building outline vectors which are due to noisy data. A cost function for the regularization process is designed based on Minimum Description Length (MDL) theory, which favours smaller deviation between a model and observation as well as orthogonal and parallel properties between polylines. Next, a new approach, called Piecewise Model Growing (PMG), is proposed for 3D PL model reconstruction using a catenary curve model. It piece-wisely grows to capture all PL points of interest and thus produces a full PL 3D model. However, the proposed method is limited to the PL scene complexity, which causes PL modeling errors such as partial, under- and over-modeling errors. To correct the incompletion of PL models, the inner and across span analysis are carried out, which leads to replace erroneous PL segments by precise PL models. The inner span analysis is performed based on the MDL theory to correct under- and over-modeling errors. The across span analysis is subsequently carried out to correct partial-modeling errors by finding start and end positions of PLs which denotes Point Of Attachment (POA). As a result, this thesis addresses not only geometrically describing building and PL objects but also dealing with noisy data which causes the incompletion of models. In the practical aspects, the results of building and PL modeling should be essential to effectively analyze a PL scene and quickly alleviate the potentially hazardous scenarios jeopardizing the PL system

Ground Profile Recovery from Aerial 3D LiDAR-based Maps

The paper presents the study and implementation of the ground detection methodology with filtration and removal of forest points from LiDAR-based 3D point cloud using the Cloth Simulation Filtering (CSF) algorithm. The methodology allows to recover a terrestrial relief and create a landscape map of a forestry region. As the proof-of-concept, we provided the outdoor flight experiment, launching a hexacopter under a mixed forestry region with sharp ground changes nearby Innopolis city (Russia), which demonstrated the encouraging results for both ground detection and methodology robustness.Comment: 8 pages, FRUCT-2019 conferenc