1,874 research outputs found
Sparsity Invariant CNNs
In this paper, we consider convolutional neural networks operating on sparse
inputs with an application to depth upsampling from sparse laser scan data.
First, we show that traditional convolutional networks perform poorly when
applied to sparse data even when the location of missing data is provided to
the network. To overcome this problem, we propose a simple yet effective sparse
convolution layer which explicitly considers the location of missing data
during the convolution operation. We demonstrate the benefits of the proposed
network architecture in synthetic and real experiments with respect to various
baseline approaches. Compared to dense baselines, the proposed sparse
convolution network generalizes well to novel datasets and is invariant to the
level of sparsity in the data. For our evaluation, we derive a novel dataset
from the KITTI benchmark, comprising 93k depth annotated RGB images. Our
dataset allows for training and evaluating depth upsampling and depth
prediction techniques in challenging real-world settings and will be made
available upon publication
Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks
Convolutional Neural Networks (CNNs) have been consistently proved
state-of-the-art results in image Super-Resolution (SR), representing an
exceptional opportunity for the remote sensing field to extract further
information and knowledge from captured data. However, most of the works
published in the literature have been focusing on the Single-Image
Super-Resolution problem so far. At present, satellite based remote sensing
platforms offer huge data availability with high temporal resolution and low
spatial resolution. In this context, the presented research proposes a novel
residual attention model (RAMS) that efficiently tackles the multi-image
super-resolution task, simultaneously exploiting spatial and temporal
correlations to combine multiple images. We introduce the mechanism of visual
feature attention with 3D convolutions in order to obtain an aware data fusion
and information extraction of the multiple low-resolution images, transcending
limitations of the local region of convolutional operations. Moreover, having
multiple inputs with the same scene, our representation learning network makes
extensive use of nestled residual connections to let flow redundant
low-frequency signals and focus the computation on more important
high-frequency components. Extensive experimentation and evaluations against
other available solutions, either for single or multi-image super-resolution,
have demonstrated that the proposed deep learning-based solution can be
considered state-of-the-art for Multi-Image Super-Resolution for remote sensing
applications
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks
Despite a rapid rise in the quality of built-in smartphone cameras, their
physical limitations - small sensor size, compact lenses and the lack of
specific hardware, - impede them to achieve the quality results of DSLR
cameras. In this work we present an end-to-end deep learning approach that
bridges this gap by translating ordinary photos into DSLR-quality images. We
propose learning the translation function using a residual convolutional neural
network that improves both color rendition and image sharpness. Since the
standard mean squared loss is not well suited for measuring perceptual image
quality, we introduce a composite perceptual error function that combines
content, color and texture losses. The first two losses are defined
analytically, while the texture loss is learned in an adversarial fashion. We
also present DPED, a large-scale dataset that consists of real photos captured
from three different phones and one high-end reflex camera. Our quantitative
and qualitative assessments reveal that the enhanced image quality is
comparable to that of DSLR-taken photos, while the methodology is generalized
to any type of digital camera
Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution
In this work, we investigate the value of uncertainty modeling in 3D
super-resolution with convolutional neural networks (CNNs). Deep learning has
shown success in a plethora of medical image transformation problems, such as
super-resolution (SR) and image synthesis. However, the highly ill-posed nature
of such problems results in inevitable ambiguity in the learning of networks.
We propose to account for intrinsic uncertainty through a per-patch
heteroscedastic noise model and for parameter uncertainty through approximate
Bayesian inference in the form of variational dropout. We show that the
combined benefits of both lead to the state-of-the-art performance SR of
diffusion MR brain images in terms of errors compared to ground truth. We
further show that the reduced error scores produce tangible benefits in
downstream tractography. In addition, the probabilistic nature of the methods
naturally confers a mechanism to quantify uncertainty over the super-resolved
output. We demonstrate through experiments on both healthy and pathological
brains the potential utility of such an uncertainty measure in the risk
assessment of the super-resolved images for subsequent clinical use.Comment: Accepted paper at MICCAI 201
Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps
This paper addresses the problem of single image depth estimation (SIDE),
focusing on improving the quality of deep neural network predictions. In a
supervised learning scenario, the quality of predictions is intrinsically
related to the training labels, which guide the optimization process. For
indoor scenes, structured-light-based depth sensors (e.g. Kinect) are able to
provide dense, albeit short-range, depth maps. On the other hand, for outdoor
scenes, LiDARs are considered the standard sensor, which comparatively provides
much sparser measurements, especially in areas further away. Rather than
modifying the neural network architecture to deal with sparse depth maps, this
article introduces a novel densification method for depth maps, using the
Hilbert Maps framework. A continuous occupancy map is produced based on 3D
points from LiDAR scans, and the resulting reconstructed surface is projected
into a 2D depth map with arbitrary resolution. Experiments conducted with
various subsets of the KITTI dataset show a significant improvement produced by
the proposed Sparse-to-Continuous technique, without the introduction of extra
information into the training stage.Comment: Accepted. (c) 2019 IEEE. Personal use of this material is permitted.
Permission from IEEE must be obtained for all other uses, in any current or
future media, including reprinting/republishing this material for advertising
or promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other work
DeepSUM: Deep Neural Network for Super-Resolution of Unregistered Multitemporal Images
Recently, convolutional neural networks (CNNs) have been successfully applied to many remote sensing problems. However, deep learning techniques for multi-image super-resolution (SR) from multitemporal unregistered imagery have received little attention so far. This article proposes a novel CNN-based technique that exploits both spatial and temporal correlations to combine multiple images. This novel framework integrates the spatial registration task directly inside the CNN, and allows one to exploit the representation learning capabilities of the network to enhance registration accuracy. The entire SR process relies on a single CNN with three main stages: shared 2-D convolutions to extract high-dimensional features from the input images; a subnetwork proposing registration filters derived from the high-dimensional feature representations; 3-D convolutions for slow fusion of the features from multiple images. The whole network can be trained end-to-end to recover a single high-resolution image from multiple unregistered low-resolution images. The method presented in this article is the winner of the PROBA-V SR challenge issued by the European Space Agency (ESA)
Permutation invariance and uncertainty in multitemporal image super-resolution
Recent advances have shown how deep neural networks can be extremely effective at super-resolving remote sensing imagery, starting from a multitemporal collection of low-resolution images. However, existing models have neglected the issue of temporal permutation, whereby the temporal ordering of the input images does not carry any relevant information for the super-resolution task and causes such models to be inefficient with the, often scarce, ground truth data that available for training. Thus, models ought not to learn feature extractors that rely on temporal ordering. In this paper, we show how building a model that is fully invariant to temporal permutation significantly improves performance and data efficiency. Moreover, we study how to quantify the uncertainty of the super-resolved image so that the final user is informed on the local quality of the product. We show how uncertainty correlates with temporal variation in the series, and how quantifying it further improves model performance. Experiments on the Proba-V challenge dataset show significant improvements over the state of the art without the need for self-ensembling, as well as improved data efficiency, reaching the performance of the challenge winner with just 25% of the training data
Deep learning for inverse problems in remote sensing: super-resolution and SAR despeckling
L'abstract è presente nell'allegato / the abstract is in the attachmen
- …