Physics-constrained Hyperspectral Data Exploitation Across Diverse Atmospheric Scenarios

Hyperspectral target detection promises new operational advantages, with increasing instrument spectral resolution and robust material discrimination. Resolving surface materials requires a fast and accurate accounting of atmospheric effects to increase detection accuracy while minimizing false alarms. This dissertation investigates deep learning methods constrained by the processes governing radiative transfer to efficiently perform atmospheric compensation on data collected by long-wave infrared (LWIR) hyperspectral sensors. These compensation methods depend on generative modeling techniques and permutation invariant neural network architectures to predict LWIR spectral radiometric quantities. The compensation algorithms developed in this work were examined from the perspective of target detection performance using collected data. These deep learning-based compensation algorithms resulted in comparable detection performance to established methods while accelerating the image processing chain by 8X

The Need for Accurate Pre-processing and Data Integration for the Application of Hyperspectral Imaging in Mineral Exploration

Die hyperspektrale Bildgebung stellt eine Schlüsseltechnologie in der nicht-invasiven Mineralanalyse dar, sei es im Labormaßstab oder als fernerkundliche Methode. Rasante Entwicklungen im Sensordesign und in der Computertechnik hinsichtlich Miniaturisierung, Bildauflösung und Datenqualität ermöglichen neue Einsatzgebiete in der Erkundung mineralischer Rohstoffe, wie die drohnen-gestützte Datenaufnahme oder digitale Aufschluss- und Bohrkernkartierung. Allgemeingültige Datenverarbeitungsroutinen fehlen jedoch meist und erschweren die Etablierung dieser vielversprechenden Ansätze. Besondere Herausforderungen bestehen hinsichtlich notwendiger radiometrischer und geometrischer Datenkorrekturen, der räumlichen Georeferenzierung sowie der Integration mit anderen Datenquellen. Die vorliegende Arbeit beschreibt innovative Arbeitsabläufe zur Lösung dieser Problemstellungen und demonstriert die Wichtigkeit der einzelnen Schritte. Sie zeigt das Potenzial entsprechend prozessierter spektraler Bilddaten für komplexe Aufgaben in Mineralexploration und Geowissenschaften.Hyperspectral imaging (HSI) is one of the key technologies in current non-invasive material analysis. Recent developments in sensor design and computer technology allow the acquisition and processing of high spectral and spatial resolution datasets. In contrast to active spectroscopic approaches such as X-ray fluorescence or laser-induced breakdown spectroscopy, passive hyperspectral reflectance measurements in the visible and infrared parts of the electromagnetic spectrum are considered rapid, non-destructive, and safe. Compared to true color or multi-spectral imagery, a much larger range and even small compositional changes of substances can be differentiated and analyzed. Applications of hyperspectral reflectance imaging can be found in a wide range of scientific and industrial fields, especially when physically inaccessible or sensitive samples and processes need to be analyzed. In geosciences, this method offers a possibility to obtain spatially continuous compositional information of samples, outcrops, or regions that might be otherwise inaccessible or too large, dangerous, or environmentally valuable for a traditional exploration at reasonable expenditure. Depending on the spectral range and resolution of the deployed sensor, HSI can provide information about the distribution of rock-forming and alteration minerals, specific chemical compounds and ions. Traditional operational applications comprise space-, airborne, and lab-scale measurements with a usually (near-)nadir viewing angle. The diversity of available sensors, in particular the ongoing miniaturization, enables their usage from a wide range of distances and viewing angles on a large variety of platforms. Many recent approaches focus on the application of hyperspectral sensors in an intermediate to close sensor-target distance (one to several hundred meters) between airborne and lab-scale, usually implying exceptional acquisition parameters. These comprise unusual viewing angles as for the imaging of vertical targets, specific geometric and radiometric distortions associated with the deployment of small moving platforms such as unmanned aerial systems (UAS), or extreme size and complexity of data created by large imaging campaigns. Accurate geometric and radiometric data corrections using established methods is often not possible. Another important challenge results from the overall variety of spatial scales, sensors, and viewing angles, which often impedes a combined interpretation of datasets, such as in a 2D geographic information system (GIS). Recent studies mostly referred to work with at least partly uncorrected data that is not able to set the results in a meaningful spatial context. These major unsolved challenges of hyperspectral imaging in mineral exploration initiated the motivation for this work. The core aim is the development of tools that bridge data acquisition and interpretation, by providing full image processing workflows from the acquisition of raw data in the field or lab, to fully corrected, validated and spatially registered at-target reflectance datasets, which are valuable for subsequent spectral analysis, image classification, or fusion in different operational environments at multiple scales. I focus on promising emerging HSI approaches, i.e.: (1) the use of lightweight UAS platforms, (2) mapping of inaccessible vertical outcrops, sometimes at up to several kilometers distance, (3) multi-sensor integration for versatile sample analysis in the near-field or lab-scale, and (4) the combination of reflectance HSI with other spectroscopic methods such as photoluminescence (PL) spectroscopy for the characterization of valuable elements in low-grade ores. In each topic, the state of the art is analyzed, tailored workflows are developed to meet key challenges and the potential of the resulting dataset is showcased on prominent mineral exploration related examples. Combined in a Python toolbox, the developed workflows aim to be versatile in regard to utilized sensors and desired applications

Performance comparison of hyperspectral target detection algorithms

This thesis performs a performance comparison on existing hyperspectral target detection algorithms. The algorithms chosen for this analysis include multiple adaptive matched filters and the physics based modeling invariant technique. The adaptive matched filter algorithms can be divided into either structured (geometrical) or unstructured (statistical) algorithms. The difference between these two categories is in the manner in which the background is characterized. The target detection procedure includes multiple pre-processing steps that are examined here as well. The effects of atmospheric compensation, dimensionality reduction, background characterization, and target subspace creation are all analyzed in terms of target detection performance. At each step of the process, techniques were chosen that consistently improved target detection performance. The best case scenario for each algorithm is used in the final comparison of performance. The results for multiple targets were computed and statistical matched filter algorithms were shown to outperform all others in a fair comparison. This fair comparison utilized a FLAASH atmospheric compensation for the matched filters that was equivalent to the physics based invariant process. The invariant technique was shown to outperform the geometric matched filters that it uses in its approach. Each of these techniques showed improvement over the SAM algorithm for three of the four targets analyzed. Multiple theories are proposed to explain the anomalous results for the most difficult target

Assessing the impact of illumination on UAV pushbroom hyperspectral imagery collected under various cloud cover conditions

The recent development of small form-factor (<6 kg), full range (400–2500 nm) pushbroom hyperspectral imaging systems (HSI) for unmanned aerial vehicles (UAV) poses a new range of opportunities for passive remote sensing applications. The flexible deployment of these UAV-HSI systems have the potential to expand the data acquisition window to acceptable (though non-ideal) atmospheric conditions. This is an important consideration for time-sensitive applications (e.g. phenology) in areas with persistent cloud cover. Since the majority of UAV studies have focused on applications with ideal illumination conditions (e.g. minimal or non-cloud cover), little is known to what extent UAV-HSI data are affected by changes in illumination conditions due to variable cloud cover. In this study, we acquired UAV pushbroom HSI (400–2500 nm) over three consecutive days with various illumination conditions (i.e. cloud cover), which were complemented with downwelling irradiance data to characterize illumination conditions and in-situ and laboratory reference panel measurements across a range of reflectivity (i.e. 2%, 10%, 18% and 50%) used to evaluate reflectance products. Using these data we address four fundamental aspects for UAV-HSI acquired under various conditions ranging from high (624.6 ± 16.63 W·m2) to low (2.5 ± 0.9 W·m2) direct irradiance: atmospheric compensation, signal-to-noise ratio (SNR), spectral vegetation indices and endmembers extraction. For instance, two atmospheric compensation methods were applied, a radiative transfer model suitable for high direct irradiance, and an Empirical Line Model (ELM) for diffuse irradiance conditions. SNR results for two distinctive vegetation classes (i.e. tree canopy vs herbaceous vegetation) reveal wavelength dependent attenuation by cloud cover, with higher SNR under high direct irradiance for canopy vegetation. Spectral vegetation index (SVIs) results revealed high variability and index dependent effects. For example, NDVI had significant differences (p < 0.05) across illumination conditions, while NDWI appeared insensitive at the canopy level. Finally, often neglected diffuse illumination conditions may be beneficial for revealing spectral features in vegetation that are obscured by the predominantly non-Lambertian reflectance encountered under high direct illumination. To our knowledge, our study is the first to use a full range pushbroom UAV sensor (400–2500 nm) for assessing illumination effects on the aforementioned variables. Our findings pave the way for understanding the advantages and limitations of ultra-high spatial resolution full range high fidelity UAV-HSI for ecological and other applications

Hyperspectral Imaging from Ground Based Mobile Platforms and Applications in Precision Agriculture

This thesis focuses on the use of line scanning hyperspectral sensors on mobile ground based platforms and applying them to agricultural applications. First this work deals with the geometric and radiometric calibration and correction of acquired hyperspectral data. When operating at low altitudes, changing lighting conditions are common and inevitable, complicating the retrieval of a surface's reflectance, which is solely a function of its physical structure and chemical composition. Therefore, this thesis contributes the evaluation of an approach to compensate for changes in illumination and obtain reflectance that is less labour intensive than traditional empirical methods. Convenient field protocols are produced that only require a representative set of illumination and reflectance spectral samples. In addition, a method for determining a line scanning camera's rigid 6 degree of freedom (DOF) offset and uncertainty with respect to a navigation system is developed, enabling accurate georegistration and sensor fusion. The thesis then applies the data captured from the platform to two different agricultural applications. The first is a self-supervised weed detection framework that allows training of a per-pixel classifier using hyperspectral data without manual labelling. The experiments support the effectiveness of the framework, rivalling classifiers trained on hand labelled training data. Then the thesis demonstrates the mapping of mango maturity using hyperspectral data on an orchard wide scale using efficient image scanning techniques, which is a world first result. A novel classification, regression and mapping pipeline is proposed to generate per tree mango maturity averages. The results confirm that maturity prediction in mango orchards is possible in natural daylight using a hyperspectral camera, despite complex micro-illumination-climates under the canopy

Hyperspectral and Hypertemporal Longwave Infrared Data Characterization

The Army Research Lab conducted a persistent imaging experiment called the Spectral and Polarimetric Imagery Collection Experiment (SPICE) in 2012 and 2013 which focused on collecting and exploiting long wave infrared hyperspectral and polarimetric imagery. A part of this dataset was made for public release for research and development purposes. This thesis investigated the hyperspectral portion of this released dataset through data characterization and scene characterization of man-made and natural objects. First, the data were contrasted with MODerate resolution atmospheric TRANsmission (MODTRAN) results and found to be comparable. Instrument noise was characterized using an in-scene black panel, and was found to be comparable with the sensor manufacturer\u27s specication. The temporal and spatial variation of certain objects in the scene were characterized. Temporal target detection was conducted on man-made objects in the scene using three target detection algorithms: spectral angle mapper (SAM), spectral matched lter (SMF) and adaptive coherence/cosine estimator (ACE). SMF produced the best results for detecting the targets when the training and testing data originated from different time periods, with a time index percentage result of 52.9%. Unsupervised and supervised classication were conducted using spectral and temporal target signatures. Temporal target signatures produced better visual classication than spectral target signature for unsupervised classication. Supervised classication yielded better results using the spectral target signatures, with a highest weighted accuracy of 99% for 7-class reference image. Four emissivity retrieval algorithms were applied on this dataset. However, the retrieved emissivities from all four methods did not represent true material emissivity and could not be used for analysis. This spectrally and temporally rich dataset enabled to conduct analysis that was not possible with other data collections. Regarding future work, applying noise-reduction techniques before applying temperature-emissivity retrieval algorithms may produce more realistic emissivity values, which could be used for target detection and material identification

Analysis of compressive sensing for hyperspectral remote sensing applications

Compressive Sensing (CS) systems capture data with fewer measurements than traditional sensors assuming that imagery is redundant and compressible in the spectral and spatial dimensions. This thesis utilizes a model of the Coded Aperture Snapshot Spectral Imager-Dual Disperser (CASSI-DD) to simulate CS measurements from traditionally sensed HyMap images. A novel reconstruction algorithm that combines spectral smoothing and spatial total variation (TV) is used to create high resolution hyperspectral imagery from the simulated CS measurements. This research examines the effect of the number of measurements, which corresponds to the percentage of physical data sampled, on the quality of simulated CS data as estimated through performance of spectral image processing algorithms. The effect of CS on the data cloud is explored through principal component analysis (PCA) and endmember extraction. The ultimate purpose of this thesis is to investigate the utility of the CS sensor model and reconstruction for various hyperspectral applications in order to identify the strengths and limitations of CS. While CS is shown to create useful imagery for visual analysis, the data cloud is altered and per-pixel spectral fidelity declines for CS reconstructions from only a small number of measurements. In some hyperspectral applications, many measurements are needed in order to obtain comparable results to traditionally sensed HSI, including atmospheric compensation and subpixel target detection. On the other hand, in hyperspectral applications where pixels must be dramatically altered in order to be misclassified, such as land classification or NDVI mapping, CS shows promise

Physics-Based Detection of Subpixel Targets in Hyperspectral Imagery

Hyperspectral imagery provides the ability to detect targets that are smaller than the size of a pixel. They provide this ability by measuring the reflection and absorption of light at different wavelengths creating a spectral signature for each pixel in the image. This spectral signature contains information about the different materials within the pixel; therefore, the challenge in subpixel target detection lies in separating the target's spectral signature from competing background signatures. Most research has approached this problem in a purely statistical manner. Our approach fuses statistical signal processing techniques with the physics of reflectance spectroscopy and radiative transfer theory. Using this approach, we provide novel algorithms for all aspects of subpixel detection from parameter estimation to threshold determination. Characterization of the target and background spectral signatures is a key part of subpixel detection. We develop an algorithm to generate target signatures based on radiative transfer theory using only the image and a reference signature without the need for calibration, weather information, or source-target-receiver geometries. For background signatures, our work identifies that even slight estimation errors in the number of background signatures can severely degrade detection performance. To this end, we present a new method to estimate the number of background signatures specifically for subpixel target detection. At the core of the dissertation is the development of two hybrid detectors which fuse spectroscopy with statistical hypothesis testing. Our results show that the hybrid detectors provide improved performance in three different ways: insensitivity to the number of background signatures, improved detection performance, and consistent performance across multiple images leading to improved receiver operating characteristic curves. Lastly, we present a novel adaptive threshold estimate via extreme value theory. The method can be used on any detector type - not just those that are constant false alarm rate (CFAR) detectors. Even on CFAR detectors our proposed method can estimate thresholds that are better than theoretical predictions due to the inherent mismatch between the CFAR model assumptions and real data. Additionally, our method works in the presence of target detections while still estimating an accurate threshold for a desired false alarm rate

Spectral image utility for target detection applications

In a wide range of applications, images convey useful information about scenes. The “utility” of an image is defined with reference to the specific task that an observer seeks to accomplish, and differs from the “fidelity” of the image, which seeks to capture the ability of the image to represent the true nature of the scene. In remote sensing of the earth, various means of characterizing the utility of satellite and airborne imagery have evolved over the years. Recent advances in the imaging modality of spectral imaging have enabled synoptic views of the earth at many finely sampled wavelengths over a broad spectral band. These advances challenge the ability of traditional earth observation image utility metrics to describe the rich information content of spectral images. Traditional approaches to image utility that are based on overhead panchromatic image interpretability by a human observer are not applicable to spectral imagery, which requires automated processing. This research establishes the context for spectral image utility by reviewing traditional approaches and current methods for describing spectral image utility. It proposes a new approach to assessing and predicting spectral image utility for the specific application of target detection. We develop a novel approach to assessing the utility of any spectral image using the target-implant method. This method is not limited by the requirements of traditional target detection performance assessment, which need ground truth and an adequate number of target pixels in the scene. The flexibility of this approach is demonstrated by assessing the utility of a wide range of real and simulated spectral imagery over a variety ii of target detection scenarios. The assessed image utility may be summarized to any desired level of specificity based on the image analysis requirements. We also present an approach to predicting spectral image utility that derives statistical parameters directly from an image and uses them to model target detection algorithm output. The image-derived predicted utility is directly comparable to the assessed utility and the accuracy of prediction is shown to improve with statistical models that capture the non-Gaussian behavior of real spectral image target detection algorithm outputs. The sensitivity of the proposed spectral image utility metric to various image chain parameters is examined in detail, revealing characteristics, requirements, and limitations that provide insight into the relative importance of parameters in the image utility. The results of these investigations lead to a better understanding of spectral image information vis-à-vis target detection performance that will hopefully prove useful to the spectral imagery analysis community and represent a step towards quantifying the ability of a spectral image to satisfy information exploitation requirements

Air Force Institute of Technology Research Report 2011

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems and Engineering Management, Operational Sciences, Mathematics, Statistics and Engineering Physics