A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Automated lithological mapping using airborne hyperspectral thermal infrared data: A case study from Anchorage Island, Antarctica

The thermal infrared portion of the electromagnetic spectrum has considerable potential for mineral and lithological mapping of the most abundant rock-forming silicates that do not display diagnostic features at visible and shortwave infrared wavelengths. Lithological mapping using visible and shortwave infrared hyperspectral data is well developed and established processing chains are available, however there is a paucity of such methodologies for hyperspectral thermal infrared data. Here we present a new fully automated processing chain for deriving lithological maps from hyperspectral thermal infrared data and test its applicability using the first ever airborne hyperspectral thermal data collected in the Antarctic. A combined airborne hyperspectral survey, targeted geological field mapping campaign and detailed mineralogical and geochemical datasets are applied to small test site in West Antarctica where the geological relationships are representative of continental margin arcs. The challenging environmental conditions and cold temperatures in the Antarctic meant that the data have a significantly lower signal to noise ratio than is usually attained from airborne hyperspectral sensors. We applied preprocessing techniques to improve the signal to noise ratio and convert the radiance images to ground leaving emissivity. Following preprocessing we developed and applied a fully automated processing chain to the hyperspectral imagery, which consists of the following six steps: (1) superpixel segmentation, (2) determine the number of endmembers, (3) extract endmembers from superpixels, (4) apply fully constrained linear unmixing, (5) generate a predictive classification map, and (6) automatically label the predictive classes to generate a lithological map. The results show that the image processing chain was successful, despite the low signal to noise ratio of the imagery; reconstruction of the hyperspectral image from the endmembers and their fractional abundances yielded a root mean square error of 0.58%. The results are encouraging with the thermal imagery allowing clear distinction between granitoid types. However, the distinction of fine grained, intermediate composition dykes is not possible due to the close geochemical similarity with the country rock

Nonparametric Detection of Nonlinearly Mixed Pixels and Endmember Estimation in Hyperspectral Images

International audienceMixing phenomena in hyperspectral images depend on a variety of factors, such as the resolution of observation devices, the properties of materials, and how these materials interact with incident light in the scene. Different parametric and nonparametric models have been considered to address hyperspectral unmixing problems. The simplest one is the linear mixing model. Nevertheless, it has been recognized that the mixing phenomena can also be nonlinear. The corresponding nonlinear analysis techniques are necessarily more challenging and complex than those employed for linear unmixing. Within this context, it makes sense to detect the nonlinearly mixed pixels in an image prior to its analysis, and then employ the simplest possible unmixing technique to analyze each pixel. In this paper, we propose a technique for detecting nonlinearly mixed pixels. The detection approach is based on the comparison of the reconstruction errors using both a Gaussian process regression model and a linear regression model. The two errors are combined into a detection statistics for which a probability density function can be reasonably approximated. We also propose an iterative endmember extraction algorithm to be employed in combination with the detection algorithm. The proposed detect-then-unmix strategy, which consists of extracting endmembers, detecting nonlinearly mixed pixels and unmixing, is tested with synthetic and real images

Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package

Spectral pixels are often a mixture of the pure spectra of the materials, called endmembers, due to the low spatial resolution of hyperspectral sensors, double scattering, and intimate mixtures of materials in the scenes. Unmixing estimates the fractional abundances of the endmembers within the pixel. Depending on the prior knowledge of endmembers, linear unmixing can be divided into three main groups: supervised, semi-supervised, and unsupervised (blind) linear unmixing. Advances in Image processing and machine learning substantially affected unmixing. This paper provides an overview of advanced and conventional unmixing approaches. Additionally, we draw a critical comparison between advanced and conventional techniques from the three categories. We compare the performance of the unmixing techniques on three simulated and two real datasets. The experimental results reveal the advantages of different unmixing categories for different unmixing scenarios. Moreover, we provide an open-source Python-based package available at https://github.com/BehnoodRasti/HySUPP to reproduce the results

Hyperspectral Image Unmixing Incorporating Adjacency Information

While the spectral information contained in hyperspectral images is rich, the spatial resolution of such images is in many cases very low. Many pixel spectra are mixtures of pure materials’ spectra and therefore need to be decomposed into their constituents. This work investigates new decomposition methods taking into account spectral, spatial and global 3D adjacency information. This allows for faster and more accurate decomposition results