123 research outputs found
Unsupervised Adversarial Depth Estimation using Cycled Generative Networks
While recent deep monocular depth estimation approaches based on supervised
regression have achieved remarkable performance, costly ground truth
annotations are required during training. To cope with this issue, in this
paper we present a novel unsupervised deep learning approach for predicting
depth maps and show that the depth estimation task can be effectively tackled
within an adversarial learning framework. Specifically, we propose a deep
generative network that learns to predict the correspondence field i.e. the
disparity map between two image views in a calibrated stereo camera setting.
The proposed architecture consists of two generative sub-networks jointly
trained with adversarial learning for reconstructing the disparity map and
organized in a cycle such as to provide mutual constraints and supervision to
each other. Extensive experiments on the publicly available datasets KITTI and
Cityscapes demonstrate the effectiveness of the proposed model and competitive
results with state of the art methods. The code and trained model are available
on https://github.com/andrea-pilzer/unsup-stereo-depthGAN.Comment: To appear in 3DV 2018. Code is available on GitHu
Self-supervised generative adverrsarial network for depth estimation in laparoscopic images
Dense depth estimation and 3D reconstruction of a surgical scene are crucial steps in computer assisted surgery. Recent work has shown that depth estimation from a stereo image pair could be solved with convolutional neural networks. However, most recent depth estimation models were trained on datasets with per-pixel ground truth. Such data is especially rare for laparoscopic imaging, making it hard to apply supervised depth estimation to real surgical applications. To overcome this limitation, we propose SADepth, a new self-supervised depth estimation method based on Generative Adversarial Networks. It consists of an encoder-decoder generator and a discriminator to incorporate geometry constraints during training. Multi-scale outputs from the generator help to solve the local minima caused by the photometric reprojection loss, while the adversarial learning improves the framework generation quality. Extensive experiments on two public datasets show that SADepth outperforms recent state-of-the-art unsupervised methods by a large margin, and reduces the gap between supervised and unsupervised depth estimation in laparoscopic images
VoloGAN: Adversarial Domain Adaptation for Synthetic Depth Data
We present VoloGAN, an adversarial domain adaptation network that translates
synthetic RGB-D images of a high-quality 3D model of a person, into RGB-D
images that could be generated with a consumer depth sensor. This system is
especially useful to generate high amount training data for single-view 3D
reconstruction algorithms replicating the real-world capture conditions, being
able to imitate the style of different sensor types, for the same high-end 3D
model database. The network uses a CycleGAN framework with a U-Net architecture
for the generator and a discriminator inspired by SIV-GAN. We use different
optimizers and learning rate schedules to train the generator and the
discriminator. We further construct a loss function that considers image
channels individually and, among other metrics, evaluates the structural
similarity. We demonstrate that CycleGANs can be used to apply adversarial
domain adaptation of synthetic 3D data to train a volumetric video generator
model having only few training samples
A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution
This paper presents a transfer domain strategy to tackle the limitations of low-resolution thermal sensors and generate higher-resolution images of reasonable quality. The proposed technique employs a CycleGAN architecture and uses a ResNet as an encoder in the generator along with an attention module and a novel loss function. The network is trained on a multi-resolution thermal image dataset acquired with three different thermal sensors. Results report better performance benchmarking results on the 2nd CVPR-PBVS-2021 thermal image super-resolution challenge than state-of-the-art methods. The code of this work is available online
- …