A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

Spectral fuzzy classification system for target recognition

The goal of this paper is to present an algorithm for terrain matching, leveraging an existing fuzzy clustering algorithm, and modifying it to its supervised version, in order to apply the algorithm to georegistration and, later on pattern recognition. Georegistration is the process of adjusting one drawing or image to the geographic location of a "known good" reference drawing, image, surface or map, The georegistration problem can be treated as a pattern recognition problem; and it can be applied to the target detection problem. The terrain matching algorithm will be based on fuzzy set theory as a very accurate method to represent the imprecision of the real world, and presented as a multicriteria decision making problem. The energy emitted and reflected by the Earth's surface has to be recorded by relatively complex remote sensing devices that have spatial, spectral and geometrical resolution constraints. Errors usually slip into the data acquisition process. Therefore, it is necessary to preprocess the remotely sensed data, prior to analyzing it (image restoration, involving the correction of distortion, degradation and noise introduced during the rendering process). In this paper we shall assume that all these problems have been solved, focusing our study on the image classification of a corrected image being close enough, both geometrically and radiometrically, to the radiant energy characteristics of the target scene. In particular, at a first stage we consider each pixel individually; and a class will be assigned to each pixel, taking into account several values measured in separate spectral bands. Then we shall describe an automatic detection system based on a previous algorithm developed in A. Del Amo et al., introducing now the fuzzy partition model proposed by A. Del Amo et al. A first phase will lead to a spectral definition of patterns; and a second phase will lead to classification and recognition. Similarity measures will then allow us to evaluate the degree to which a pixel can be associated to each pattern, or determine if a pattern is similar enough to a subimage of the main image, to establish that a target we are looking for can be found on that image