A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Breaking new ground in mapping human settlements from space -The Global Urban Footprint-

Today 7.2 billion people inhabit the Earth and by 2050 this number will have risen to around nine billion, of which about 70 percent will be living in cities. Hence, it is essential to understand drivers, dynamics, and impacts of the human settlements development. A key component in this context is the availability of an up-to-date and spatially consistent map of the location and distribution of human settlements. It is here that the Global Urban Footprint (GUF) raster map can make a valuable contribution. The new global GUF binary settlement mask shows a so far unprecedented spatial resolution of 0.4 arcsec ($\sim12 m$) that provides - for the first time - a complete picture of the entirety of urban and rural settlements. The GUF has been derived by means of a fully automated processing framework - the Urban Footprint Processor (UFP) - that was used to analyze a global coverage of more than 180,000 TanDEM-X and TerraSAR-X radar images with 3m ground resolution collected in 2011-2012. Various quality assessment studies to determine the absolute GUF accuracy based on ground truth data on the one hand and the relative accuracies compared to established settlements maps on the other hand, clearly indicate the added value of the new global GUF layer, in particular with respect to the representation of rural settlement patterns. Generally, the GUF layer achieves an overall absolute accuracy of about 85\%, with observed minima around 65\% and maxima around 98 \%. The GUF will be provided open and free for any scientific use in the full resolution and for any non-profit (but also non-scientific) use in a generalized version of 2.8 arcsec ($\sim84m$). Therewith, the new GUF layer can be expected to break new ground with respect to the analysis of global urbanization and peri-urbanization patterns, population estimation or vulnerability assessment

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Learning a Dilated Residual Network for SAR Image Despeckling

In this paper, to break the limit of the traditional linear models for synthetic aperture radar (SAR) image despeckling, we propose a novel deep learning approach by learning a non-linear end-to-end mapping between the noisy and clean SAR images with a dilated residual network (SAR-DRN). SAR-DRN is based on dilated convolutions, which can both enlarge the receptive field and maintain the filter size and layer depth with a lightweight structure. In addition, skip connections and residual learning strategy are added to the despeckling model to maintain the image details and reduce the vanishing gradient problem. Compared with the traditional despeckling methods, the proposed method shows superior performance over the state-of-the-art methods on both quantitative and visual assessments, especially for strong speckle noise.Comment: 18 pages, 13 figures, 7 table

DoPAMINE: Double-sided Masked CNN for Pixel Adaptive Multiplicative Noise Despeckling

We propose DoPAMINE, a new neural network based multiplicative noise despeckling algorithm. Our algorithm is inspired by Neural AIDE (N-AIDE), which is a recently proposed neural adaptive image denoiser. While the original N-AIDE was designed for the additive noise case, we show that the same framework, i.e., adaptively learning a network for pixel-wise affine denoisers by minimizing an unbiased estimate of MSE, can be applied to the multiplicative noise case as well. Moreover, we derive a double-sided masked CNN architecture which can control the variance of the activation values in each layer and converge fast to high denoising performance during supervised training. In the experimental results, we show our DoPAMINE possesses high adaptivity via fine-tuning the network parameters based on the given noisy image and achieves significantly better despeckling results compared to SAR-DRN, a state-of-the-art CNN-based algorithm.Comment: AAAI 2019 Camera Ready Versio