7 research outputs found
Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps
This paper addresses the problem of single image depth estimation (SIDE),
focusing on improving the quality of deep neural network predictions. In a
supervised learning scenario, the quality of predictions is intrinsically
related to the training labels, which guide the optimization process. For
indoor scenes, structured-light-based depth sensors (e.g. Kinect) are able to
provide dense, albeit short-range, depth maps. On the other hand, for outdoor
scenes, LiDARs are considered the standard sensor, which comparatively provides
much sparser measurements, especially in areas further away. Rather than
modifying the neural network architecture to deal with sparse depth maps, this
article introduces a novel densification method for depth maps, using the
Hilbert Maps framework. A continuous occupancy map is produced based on 3D
points from LiDAR scans, and the resulting reconstructed surface is projected
into a 2D depth map with arbitrary resolution. Experiments conducted with
various subsets of the KITTI dataset show a significant improvement produced by
the proposed Sparse-to-Continuous technique, without the introduction of extra
information into the training stage.Comment: Accepted. (c) 2019 IEEE. Personal use of this material is permitted.
Permission from IEEE must be obtained for all other uses, in any current or
future media, including reprinting/republishing this material for advertising
or promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other work
Guided Filters for Depth Image Enhancement
This thesis proposes an approach utilizing guided techniques to refine depth images. Given a depth image and a color image of the same resolution, we can utilize the color image as a guide to improve the accuracy of the depth image, by smoothing out edges and removing holes as much as possible. This is done utilizing a guided filter, which solves an optimization problem relating the depth and color image to smooth and refine the depth image. These guided filters are linear-time and much faster than other state-of-the-art methods, while producing comparable results. We also integrate an existing guided inpainting model, further removing holes and improving the depth map. In this thesis, we show the application of guided filters to the depth refinement problem, utilize a guided inpainting model to fill in any holes that may arise in the depth image, as well as extend the filter out to the temporal domain to handle temporal flickering. This is done via an extension of existing optical-flow methods to compute a weighted average of the previous and next neighbors. We also demonstrate a few experimental results on real-time video to show that this method has viability in consumer depth applications. We demonstrate results on both datasets and real video to show the accuracy of our method.Ope
An initial matching and mapping for dense 3D object tracking in augmented reality applications
Augmented Reality (AR) applications rely on efficient and robust methods of tracking. One type of tracking uses dense 3D point data representations of the object to track. As opposed to sparse, dense tracking approaches are highly accurate and precise by considering all of the available data from a camera. A major challenge to dense tracking is that it requires a rough initial matching and mapping to begin. A matching means that from a known object, we can determine the object exists in the scene, and a mapping means that we can identify the position and orientation of an object with respect to the camera. Current methods to provide the initial matching and mapping require the user to manually input parameters, or wait an extended amount of time for a brute force automatic approach.
The research presented in this thesis develops an automatic initial matching and mapping for dense tracking for AR, facilitating natural AR systems that track 3D objects. To do this, an existing offline method for registration of ideal 3D object point sets is proposed as a starting point. The method is improved and optimized in four steps to address the requirements and challenges for dense tracking in AR with a noisy consumer sensor. A series of experiments verifies the suitability of the optimizations, using increasingly large and more complex scene point clouds, and the results are presented