A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Advances in multispectral and hyperspectral imaging for archaeology and art conservation

Multispectral imaging has been applied to the field of art conservation and art history since the early 1990s. It is attractive as a noninvasive imaging technique because it is fast and hence capable of imaging large areas of an object giving both spatial and spectral information. This paper gives an overview of the different instrumental designs, image processing techniques and various applications of multispectral and hyperspectral imaging to art conservation, art history and archaeology. Recent advances in the development of remote and versatile multispectral and hyperspectral imaging as well as techniques in pigment identification will be presented. Future prospects including combination of spectral imaging with other noninvasive imaging and analytical techniques will be discussed

Trying to break new ground in aerial archaeology

Aerial reconnaissance continues to be a vital tool for landscape-oriented archaeological research. Although a variety of remote sensing platforms operate within the earth’s atmosphere, the majority of aerial archaeological information is still derived from oblique photographs collected during observer-directed reconnaissance flights, a prospection approach which has dominated archaeological aerial survey for the past century. The resulting highly biased imagery is generally catalogued in sub-optimal (spatial) databases, if at all, after which a small selection of images is orthorectified and interpreted. For decades, this has been the standard approach. Although many innovations, including digital cameras, inertial units, photogrammetry and computer vision algorithms, geographic(al) information systems and computing power have emerged, their potential has not yet been fully exploited in order to re-invent and highly optimise this crucial branch of landscape archaeology. The authors argue that a fundamental change is needed to transform the way aerial archaeologists approach data acquisition and image processing. By addressing the very core concepts of geographically biased aerial archaeological photographs and proposing new imaging technologies, data handling methods and processing procedures, this paper gives a personal opinion on how the methodological components of aerial archaeology, and specifically aerial archaeological photography, should evolve during the next decade if developing a more reliable record of our past is to be our central aim. In this paper, a possible practical solution is illustrated by outlining a turnkey aerial prospection system for total coverage survey together with a semi-automated back-end pipeline that takes care of photograph correction and image enhancement as well as the management and interpretative mapping of the resulting data products. In this way, the proposed system addresses one of many bias issues in archaeological research: the bias we impart to the visual record as a result of selective coverage. While the total coverage approach outlined here may not altogether eliminate survey bias, it can vastly increase the amount of useful information captured during a single reconnaissance flight while mitigating the discriminating effects of observer-based, on-the-fly target selection. Furthermore, the information contained in this paper should make it clear that with current technology it is feasible to do so. This can radically alter the basis for aerial prospection and move landscape archaeology forward, beyond the inherently biased patterns that are currently created by airborne archaeological prospection

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensin

DEVELOPING INNOVATIVE SPECTRAL AND MACHINE LEARNING METHODS FOR MINERAL AND LITHOLOGICAL CLASSIFICATION USING MULTI-SENSOR DATASETS

The sustainable exploration of mineral resources plays a significant role in the economic development of any nation. The lithological maps and surface mineral distribution can be vital baseline data to narrow down the geochemical and geophysical analysis potential areas. This study developed innovative spectral and Machine Learning (ML) methods for mineral and lithological classification. Multi-sensor datasets such as Airborne Visible/Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG), Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), Advanced Land Observing (ALOS) Phased Array type L-band Synthetic Aperture Radar (PALSAR), Sentinel-1, and Digital Elevation Model (DEM) were utilized. The study mapped the hydrothermal alteration minerals derived from Spectral Mapping Methods (SMMs), including Spectral Angle Mapper (SAM), Spectral Information Divergence (SID), and SIDSAMtan using high-resolution AVIRIS-NG hyperspectral data in the Hutti-Maski area (India). The SIDSAMtan outperforms SID and SAM in mineral mapping. A spectral similarity matrix of target and non-target classes based optimum threshold selection was developed to implement the SMMs successfully. Three new effective SMMs such as Dice Spectral Similarity Coefficient (DSSC), Kumar-Johnson Spectral Similarity Coefficient (KJSSC), and their hybrid, i.e., KJDSSCtan has been proposed, which outperforms the existing SMMs (i.e., SAM, SID, and SIDSAMtan) in spectral discrimination of spectrally similar minerals. The developed optimum threshold selection and proposed SMMs are recommended for accurate mineral mapping using hyperspectral data. An integrated spectral enhancement and ML methods have been developed to perform automated lithological classification using AVIRIS-NG hyperspectral data. The Support Vector Machine (SVM) outperforms the Random Forest (RF) and Linear Discriminant Analysis (LDA) in lithological classification. The performance of SVM also shows the least sensitivity to the number and uncertainty of training datasets. This study proposed a multi-sensor datasets-based optimal integration of spectral, morphological, and textural characteristics of rocks for accurate lithological classification using ML models. Different input features, such as (a) spectral, (b) spectral and transformed spectral, (c) spectral and morphological, (d) spectral and textural, and (e) optimum hybrid, were evaluated for lithological classification. The developed approach has been assessed in the Chattarpur area (India) consists of similar spectral characteristics and poorly exposed rocks, weathered, and partially vegetated terrain. The optimal hybrid input features outperform other input features to accurately classify different rock types using the SVM and RF models, which is ~15% higher than as obtained using spectral input features alone. The developed integrated approach of spectral enhancement and ML algorithms, and a multi-sensor datasets-based optimal integration of spectral, morphological, and textural characteristics of rocks, are recommended for accurate lithological classification. The developed methods can be effectively utilized in other remote sensing applications, such as vegetation/forest mapping and soil classification