13,472 research outputs found
UAMD-Net: A Unified Adaptive Multimodal Neural Network for Dense Depth Completion
Depth prediction is a critical problem in robotics applications especially
autonomous driving. Generally, depth prediction based on binocular stereo
matching and fusion of monocular image and laser point cloud are two mainstream
methods. However, the former usually suffers from overfitting while building
cost volume, and the latter has a limited generalization due to the lack of
geometric constraint. To solve these problems, we propose a novel multimodal
neural network, namely UAMD-Net, for dense depth completion based on fusion of
binocular stereo matching and the weak constrain from the sparse point clouds.
Specifically, the sparse point clouds are converted to sparse depth map and
sent to the multimodal feature encoder (MFE) with binocular image, constructing
a cross-modal cost volume. Then, it will be further processed by the multimodal
feature aggregator (MFA) and the depth regression layer. Furthermore, the
existing multimodal methods ignore the problem of modal dependence, that is,
the network will not work when a certain modal input has a problem. Therefore,
we propose a new training strategy called Modal-dropout which enables the
network to be adaptively trained with multiple modal inputs and inference with
specific modal inputs. Benefiting from the flexible network structure and
adaptive training method, our proposed network can realize unified training
under various modal input conditions. Comprehensive experiments conducted on
KITTI depth completion benchmark demonstrate that our method produces robust
results and outperforms other state-of-the-art methods.Comment: 11 pages, 4 figure
- …