8,792 research outputs found
Guided Stereo Matching
Stereo is a prominent technique to infer dense depth maps from images, and
deep learning further pushed forward the state-of-the-art, making end-to-end
architectures unrivaled when enough data is available for training. However,
deep networks suffer from significant drops in accuracy when dealing with new
environments. Therefore, in this paper, we introduce Guided Stereo Matching, a
novel paradigm leveraging a small amount of sparse, yet reliable depth
measurements retrieved from an external source enabling to ameliorate this
weakness. The additional sparse cues required by our method can be obtained
with any strategy (e.g., a LiDAR) and used to enhance features linked to
corresponding disparity hypotheses. Our formulation is general and fully
differentiable, thus enabling to exploit the additional sparse inputs in
pre-trained deep stereo networks as well as for training a new instance from
scratch. Extensive experiments on three standard datasets and two
state-of-the-art deep architectures show that even with a small set of sparse
input cues, i) the proposed paradigm enables significant improvements to
pre-trained networks. Moreover, ii) training from scratch notably increases
accuracy and robustness to domain shifts. Finally, iii) it is suited and
effective even with traditional stereo algorithms such as SGM.Comment: CVPR 201
A Critical Review of Deep Learning-Based Multi-Sensor Fusion Techniques
In this review, we provide a detailed coverage of multi-sensor fusion techniques that use RGB stereo images and a sparse LiDAR-projected depth map as input data to output a dense depth map prediction. We cover state-of-the-art fusion techniques which, in recent years, have been deep learning-based methods that are end-to-end trainable. We then conduct a comparative evaluation of the state-of-the-art techniques and provide a detailed analysis of their strengths and limitations as well as the applications they are best suited for
Improved Neural Radiance Fields Using Pseudo-depth and Fusion
Since the advent of Neural Radiance Fields, novel view synthesis has received
tremendous attention. The existing approach for the generalization of radiance
field reconstruction primarily constructs an encoding volume from nearby source
images as additional inputs. However, these approaches cannot efficiently
encode the geometric information of real scenes with various scale
objects/structures. In this work, we propose constructing multi-scale encoding
volumes and providing multi-scale geometry information to NeRF models. To make
the constructed volumes as close as possible to the surfaces of objects in the
scene and the rendered depth more accurate, we propose to perform depth
prediction and radiance field reconstruction simultaneously. The predicted
depth map will be used to supervise the rendered depth, narrow the depth range,
and guide points sampling. Finally, the geometric information contained in
point volume features may be inaccurate due to occlusion, lighting, etc. To
this end, we propose enhancing the point volume feature from depth-guided
neighbor feature fusion. Experiments demonstrate the superior performance of
our method in both novel view synthesis and dense geometry modeling without
per-scene optimization
- …