991 research outputs found
Unsupervised Adversarial Depth Estimation using Cycled Generative Networks
While recent deep monocular depth estimation approaches based on supervised
regression have achieved remarkable performance, costly ground truth
annotations are required during training. To cope with this issue, in this
paper we present a novel unsupervised deep learning approach for predicting
depth maps and show that the depth estimation task can be effectively tackled
within an adversarial learning framework. Specifically, we propose a deep
generative network that learns to predict the correspondence field i.e. the
disparity map between two image views in a calibrated stereo camera setting.
The proposed architecture consists of two generative sub-networks jointly
trained with adversarial learning for reconstructing the disparity map and
organized in a cycle such as to provide mutual constraints and supervision to
each other. Extensive experiments on the publicly available datasets KITTI and
Cityscapes demonstrate the effectiveness of the proposed model and competitive
results with state of the art methods. The code and trained model are available
on https://github.com/andrea-pilzer/unsup-stereo-depthGAN.Comment: To appear in 3DV 2018. Code is available on GitHu
Recurrent Scene Parsing with Perspective Understanding in the Loop
Objects may appear at arbitrary scales in perspective images of a scene,
posing a challenge for recognition systems that process images at a fixed
resolution. We propose a depth-aware gating module that adaptively selects the
pooling field size in a convolutional network architecture according to the
object scale (inversely proportional to the depth) so that small details are
preserved for distant objects while larger receptive fields are used for those
nearby. The depth gating signal is provided by stereo disparity or estimated
directly from monocular input. We integrate this depth-aware gating into a
recurrent convolutional neural network to perform semantic segmentation. Our
recurrent module iteratively refines the segmentation results, leveraging the
depth and semantic predictions from the previous iterations.
Through extensive experiments on four popular large-scale RGB-D datasets, we
demonstrate this approach achieves competitive semantic segmentation
performance with a model which is substantially more compact. We carry out
extensive analysis of this architecture including variants that operate on
monocular RGB but use depth as side-information during training, unsupervised
gating as a generic attentional mechanism, and multi-resolution gating. We find
that gated pooling for joint semantic segmentation and depth yields
state-of-the-art results for quantitative monocular depth estimation
Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction
Multi-scale representations deeply learned via convolutional neural networks
have shown tremendous importance for various pixel-level prediction problems.
In this paper we present a novel approach that advances the state of the art on
pixel-level prediction in a fundamental aspect, i.e. structured multi-scale
features learning and fusion. In contrast to previous works directly
considering multi-scale feature maps obtained from the inner layers of a
primary CNN architecture, and simply fusing the features with weighted
averaging or concatenation, we propose a probabilistic graph attention network
structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs)
model for learning and fusing multi-scale representations in a principled
manner. In order to further improve the learning capacity of the network
structure, we propose to exploit feature dependant conditional kernels within
the deep probabilistic framework. Extensive experiments are conducted on four
publicly available datasets (i.e. BSDS500, NYUD-V2, KITTI, and Pascal-Context)
and on three challenging pixel-wise prediction problems involving both discrete
and continuous labels (i.e. monocular depth estimation, object contour
prediction, and semantic segmentation). Quantitative and qualitative results
demonstrate the effectiveness of the proposed latent AG-CRF model and the
overall probabilistic graph attention network with feature conditional kernels
for structured feature learning and pixel-wise prediction.Comment: Regular paper accepted at TPAMI 2020. arXiv admin note: text overlap
with arXiv:1801.0052
- …