156,277 research outputs found
Learning to Synthesize a 4D RGBD Light Field from a Single Image
We present a machine learning algorithm that takes as input a 2D RGB image
and synthesizes a 4D RGBD light field (color and depth of the scene in each ray
direction). For training, we introduce the largest public light field dataset,
consisting of over 3300 plenoptic camera light fields of scenes containing
flowers and plants. Our synthesis pipeline consists of a convolutional neural
network (CNN) that estimates scene geometry, a stage that renders a Lambertian
light field using that geometry, and a second CNN that predicts occluded rays
and non-Lambertian effects. Our algorithm builds on recent view synthesis
methods, but is unique in predicting RGBD for each light field ray and
improving unsupervised single image depth estimation by enforcing consistency
of ray depths that should intersect the same scene point. Please see our
supplementary video at https://youtu.be/yLCvWoQLnmsComment: International Conference on Computer Vision (ICCV) 201
Unsupervised Cross-spectral Stereo Matching by Learning to Synthesize
Unsupervised cross-spectral stereo matching aims at recovering disparity
given cross-spectral image pairs without any supervision in the form of ground
truth disparity or depth. The estimated depth provides additional information
complementary to individual semantic features, which can be helpful for other
vision tasks such as tracking, recognition and detection. However, there are
large appearance variations between images from different spectral bands, which
is a challenge for cross-spectral stereo matching. Existing deep unsupervised
stereo matching methods are sensitive to the appearance variations and do not
perform well on cross-spectral data. We propose a novel unsupervised
cross-spectral stereo matching framework based on image-to-image translation.
First, a style adaptation network transforms images across different spectral
bands by cycle consistency and adversarial learning, during which appearance
variations are minimized. Then, a stereo matching network is trained with image
pairs from the same spectra using view reconstruction loss. At last, the
estimated disparity is utilized to supervise the spectral-translation network
in an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is
proposed to improve the robustness of spectral translation. Our method can
tackle appearance variations and enhance the robustness of unsupervised
cross-spectral stereo matching. Experimental results show that our method
achieves good performance without using depth supervision or explicit semantic
information.Comment: accepted by AAAI-1
- …