3 research outputs found
DeepSUM++: Non-local Deep Neural Network for Super-Resolution of Unregistered Multitemporal Images
Deep learning methods for super-resolution of a remote sensing scene from
multiple unregistered low-resolution images have recently gained attention
thanks to a challenge proposed by the European Space Agency. This paper
presents an evolution of the winner of the challenge, showing how incorporating
non-local information in a convolutional neural network allows to exploit
self-similar patterns that provide enhanced regularization of the
super-resolution problem. Experiments on the dataset of the challenge show
improved performance over the state-of-the-art, which does not exploit
non-local information.Comment: arXiv admin note: text overlap with arXiv:1907.0649
Permutation invariance and uncertainty in multitemporal image super-resolution
Recent advances have shown how deep neural networks can be extremely effective at super-resolving remote sensing imagery, starting from a multitemporal collection of low-resolution images. However, existing models have neglected the issue of temporal permutation, whereby the temporal ordering of the input images does not carry any relevant information for the super-resolution task and causes such models to be inefficient with the, often scarce, ground truth data that available for training. Thus, models ought not to learn feature extractors that rely on temporal ordering. In this paper, we show how building a model that is fully invariant to temporal permutation significantly improves performance and data efficiency. Moreover, we study how to quantify the uncertainty of the super-resolved image so that the final user is informed on the local quality of the product. We show how uncertainty correlates with temporal variation in the series, and how quantifying it further improves model performance. Experiments on the Proba-V challenge dataset show significant improvements over the state of the art without the need for self-ensembling, as well as improved data efficiency, reaching the performance of the challenge winner with just 25% of the training data
Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks
Convolutional Neural Networks (CNNs) have been consistently proved
state-of-the-art results in image Super-Resolution (SR), representing an
exceptional opportunity for the remote sensing field to extract further
information and knowledge from captured data. However, most of the works
published in the literature have been focusing on the Single-Image
Super-Resolution problem so far. At present, satellite based remote sensing
platforms offer huge data availability with high temporal resolution and low
spatial resolution. In this context, the presented research proposes a novel
residual attention model (RAMS) that efficiently tackles the multi-image
super-resolution task, simultaneously exploiting spatial and temporal
correlations to combine multiple images. We introduce the mechanism of visual
feature attention with 3D convolutions in order to obtain an aware data fusion
and information extraction of the multiple low-resolution images, transcending
limitations of the local region of convolutional operations. Moreover, having
multiple inputs with the same scene, our representation learning network makes
extensive use of nestled residual connections to let flow redundant
low-frequency signals and focus the computation on more important
high-frequency components. Extensive experimentation and evaluations against
other available solutions, either for single or multi-image super-resolution,
have demonstrated that the proposed deep learning-based solution can be
considered state-of-the-art for Multi-Image Super-Resolution for remote sensing
applications