A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

The Need for Accurate Pre-processing and Data Integration for the Application of Hyperspectral Imaging in Mineral Exploration

Die hyperspektrale Bildgebung stellt eine Schlüsseltechnologie in der nicht-invasiven Mineralanalyse dar, sei es im Labormaßstab oder als fernerkundliche Methode. Rasante Entwicklungen im Sensordesign und in der Computertechnik hinsichtlich Miniaturisierung, Bildauflösung und Datenqualität ermöglichen neue Einsatzgebiete in der Erkundung mineralischer Rohstoffe, wie die drohnen-gestützte Datenaufnahme oder digitale Aufschluss- und Bohrkernkartierung. Allgemeingültige Datenverarbeitungsroutinen fehlen jedoch meist und erschweren die Etablierung dieser vielversprechenden Ansätze. Besondere Herausforderungen bestehen hinsichtlich notwendiger radiometrischer und geometrischer Datenkorrekturen, der räumlichen Georeferenzierung sowie der Integration mit anderen Datenquellen. Die vorliegende Arbeit beschreibt innovative Arbeitsabläufe zur Lösung dieser Problemstellungen und demonstriert die Wichtigkeit der einzelnen Schritte. Sie zeigt das Potenzial entsprechend prozessierter spektraler Bilddaten für komplexe Aufgaben in Mineralexploration und Geowissenschaften.Hyperspectral imaging (HSI) is one of the key technologies in current non-invasive material analysis. Recent developments in sensor design and computer technology allow the acquisition and processing of high spectral and spatial resolution datasets. In contrast to active spectroscopic approaches such as X-ray fluorescence or laser-induced breakdown spectroscopy, passive hyperspectral reflectance measurements in the visible and infrared parts of the electromagnetic spectrum are considered rapid, non-destructive, and safe. Compared to true color or multi-spectral imagery, a much larger range and even small compositional changes of substances can be differentiated and analyzed. Applications of hyperspectral reflectance imaging can be found in a wide range of scientific and industrial fields, especially when physically inaccessible or sensitive samples and processes need to be analyzed. In geosciences, this method offers a possibility to obtain spatially continuous compositional information of samples, outcrops, or regions that might be otherwise inaccessible or too large, dangerous, or environmentally valuable for a traditional exploration at reasonable expenditure. Depending on the spectral range and resolution of the deployed sensor, HSI can provide information about the distribution of rock-forming and alteration minerals, specific chemical compounds and ions. Traditional operational applications comprise space-, airborne, and lab-scale measurements with a usually (near-)nadir viewing angle. The diversity of available sensors, in particular the ongoing miniaturization, enables their usage from a wide range of distances and viewing angles on a large variety of platforms. Many recent approaches focus on the application of hyperspectral sensors in an intermediate to close sensor-target distance (one to several hundred meters) between airborne and lab-scale, usually implying exceptional acquisition parameters. These comprise unusual viewing angles as for the imaging of vertical targets, specific geometric and radiometric distortions associated with the deployment of small moving platforms such as unmanned aerial systems (UAS), or extreme size and complexity of data created by large imaging campaigns. Accurate geometric and radiometric data corrections using established methods is often not possible. Another important challenge results from the overall variety of spatial scales, sensors, and viewing angles, which often impedes a combined interpretation of datasets, such as in a 2D geographic information system (GIS). Recent studies mostly referred to work with at least partly uncorrected data that is not able to set the results in a meaningful spatial context. These major unsolved challenges of hyperspectral imaging in mineral exploration initiated the motivation for this work. The core aim is the development of tools that bridge data acquisition and interpretation, by providing full image processing workflows from the acquisition of raw data in the field or lab, to fully corrected, validated and spatially registered at-target reflectance datasets, which are valuable for subsequent spectral analysis, image classification, or fusion in different operational environments at multiple scales. I focus on promising emerging HSI approaches, i.e.: (1) the use of lightweight UAS platforms, (2) mapping of inaccessible vertical outcrops, sometimes at up to several kilometers distance, (3) multi-sensor integration for versatile sample analysis in the near-field or lab-scale, and (4) the combination of reflectance HSI with other spectroscopic methods such as photoluminescence (PL) spectroscopy for the characterization of valuable elements in low-grade ores. In each topic, the state of the art is analyzed, tailored workflows are developed to meet key challenges and the potential of the resulting dataset is showcased on prominent mineral exploration related examples. Combined in a Python toolbox, the developed workflows aim to be versatile in regard to utilized sensors and desired applications