An end-to-end hyperspectral scene simulator with alternate adjacency effect models and its comparison with cameoSim

In this research, we developed a new rendering-based end to end Hyperspectral scene simulator CHIMES (Cranfield Hyperspectral Image Modelling and Evaluation System), which generates nadir images of passively illuminated 3-D outdoor scenes in Visible, Near Infrared (NIR) and Short-Wave Infrared (SWIR) regions, ranging from 360 nm to 2520 nm. MODTRAN TM (MODerate resolution TRANsmission), is used to generate the sky-dome environment map which includes sun and sky radiance along with the polarisation effect of the sky due to Rayleigh scattering. Moreover, we perform path tracing and implement ray interaction with medium and volumetric backscattering at rendering time to model the adjacency effect. We propose two variants of adjacency models, the first one incorporates a single spectral albedo as the averaged background of the scene, this model is called the Background One-Spectra Adjacency Effect Model (BOAEM), which is a CameoSim like model created for performance comparison. The second model calculates background albedo from a pixel’s neighbourhood, whose size depends on the air volume between sensor and target, and differential air density up to sensor altitude. Average background reflectance of all neighbourhood pixel is computed at rendering time for estimating the total upwelled scattered radiance, by volumetric scattering. This model is termed the Texture-Spectra Incorporated Adjacency Effect Model (TIAEM). Moreover, for estimating the underlying atmospheric condition MODTRAN is run with varying aerosol optical thickness and its total ground reflected radiance (TGRR) is compared with TGRR of known in-scene material. The Goodness of fit is evaluated in each iteration, and MODTRAN’s output with the best fit is selected. We perform a tri-modal validation of simulators on a real hyperspectral scene by varying atmospheric condition, terrain surface models and proposed variants of adjacency models. We compared results of our model with Lockheed Martin’s well-established scene simulator CameoSim and acquired Ground Truth (GT) by Hyspex cameras. In clear-sky conditions, both models of CHIMES and CameoSim are in close agreement, however, in searched overcast conditions CHIMES BOAEM is shown to perform better than CameoSim in terms of ℓ1 -norm error of the whole scene with respect to GT. TIAEM produces better radiance shape and covariance of background statistics with respect to Ground Truth (GT), which is key to good target detection performance. We also report that the results of CameoSim have a many-fold higher error for the same scene when the flat surface terrain is replaced with a Digital Elevation Model (DEM) based rugged one

Signature Simulation and Characterization of Mixed Solids in the Visible and Thermal Regimes

Solid target signatures vary due to geometry, chemical composition and scene radiometry. Although radiative transfer models and function-fit physical models may describe certain targets in limited depth, the ability to incorporate all three of these signature variables is difficult. This work describes a method to simulate the transient signatures of mixed solids and soils by first considering scene geometry that was synthetically created using 3-d physics engines. Through the assignment of spectral data from the Nonconventional Exploitation Factors Data System (NEFDS) and other libraries, synthetic scenes are represented as a chemical mixture of particles. Finally, first principles radiometry is modeled using the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model. With DIRSIG, radiometric and sensing conditions were systematically manipulated to produce goniometric signatures. The implementation of this virtual goniometer allows users to examine how a target bidirectional reflectance function (BRDF) and directional emissivity will change with geometry, composition and illumination direction. The tool described provides geometry flexibility that is unmatched by radiative transfer models. It delivers a discrete method to avoid the significant cost of time and treasure associated with hardware based goniometric data collections

Hyperspectral Imaging for Real-Time Unmanned Aerial Vehicle Maritime Target Detection

The hyperspectral cameras use has been increasing over the past years, driven by the exponential growth of the computational systems power. The capability of acquiring multiple spectre wavelengths benefits the increase of the hyperspectral systems range of applications. However, until now, most hyperspectral systems are used in posprocessing and do not allow to take full advantage of the system capabilities. There is a recent trend to be able to use hyperspectral systems in real-time. Given the recent problems in European Union borders due to irregular immigration and drug smuggling, there is the need to develop novel autonomous surveillance systems that can work on these scenarios. This thesis addresses the scenario of using hyperspectral imaging systems for maritime target detection using unmanned aerial vehicles. Specifically, by working in the creation of a hyperspectral real-time data processing system pipeline. In our work, we develop a boresight calibration method that allows to calibrate the position of the navigation sensor related to the camera imaging sensor, and improve substantially the accuracy of the target geo-reference. We also develop a novel method of distinguish targets (boats) from their dominant background. With this application our system is able to only select relevant information to send to a remote station on the ground, thus making it suitable to be installed in an actual unmanned maritime surveillance system.A utilização de câmaras hiperespectrais tem vindo a aumentar nos últimos anos, motivada pelo crescimento exponencial da capacidade de processamento dos mais recentes sistemas computacionais. A sua aptidão para observar múltiplos comprimentos de onda beneficia aplicações em diferentes campos de atividade. No entanto, a maior parte das aplicações com câmaras hiperespectrais são realizadas em pós-processamento, não aproveitando totalmente as capacidades destes sistemas. Existe uma necessidade emergente de detetar mais características sobre o cenário que está a ser observado, incentivando o desenvolvimento de sistemas hiperespectrais capazes de adquirir e processar informação em tempo-real. Face aos mais recentes problemas de emigração e contrabando ilegal na União Europeia, surge a necessidade da realização de vigilância autónoma capaz de adquirir o máximo de informação possível sobre os meios envolventes presentes num dado percurso. E neste contexto que se insere a dissertação que visa a criação é implementação de um sistema hiperespectral em tempo-real. Para construir o sistema, foi necessário dividir o problema em diferentes etapas. Iniciou-se por um estudo detalhado dos sistemas hiperespectrais, desenvolvendo um método de calibração dos ângulos de boresight, que permitiu calibrar a relação entre o sistema de posicionamento e navegação da câmara hiperespectral e o sensor imagem. Esta calibração, permite numa fase posterior geo-referenciar os alvos com maior precisão. Posteriormente, foi criada uma pipeline de processamento, que permite analisar os espectros obtidos, distinguindo os alvos do cenário onde estão inseridos. Após a deteção dos alvos, procede-se `a sua geo-referenciação, de forma a obter as coordenadas UTM do alvo. Toda a informação obtida sobre o alvo e a sua posição é enviada para uma estacão em terra, de forma a ser validada por um humano. Para tal, foi também desenvolvida a metodologia de envio, para selecionar a informação a enviar apenas à mais relevante

Physics-constrained Hyperspectral Data Exploitation Across Diverse Atmospheric Scenarios

Hyperspectral target detection promises new operational advantages, with increasing instrument spectral resolution and robust material discrimination. Resolving surface materials requires a fast and accurate accounting of atmospheric effects to increase detection accuracy while minimizing false alarms. This dissertation investigates deep learning methods constrained by the processes governing radiative transfer to efficiently perform atmospheric compensation on data collected by long-wave infrared (LWIR) hyperspectral sensors. These compensation methods depend on generative modeling techniques and permutation invariant neural network architectures to predict LWIR spectral radiometric quantities. The compensation algorithms developed in this work were examined from the perspective of target detection performance using collected data. These deep learning-based compensation algorithms resulted in comparable detection performance to established methods while accelerating the image processing chain by 8X

DragonflEYE: a passive approach to aerial collision sensing

"This dissertation describes the design, development and test of a passive wide-field optical aircraft collision sensing instrument titled 'DragonflEYE'. Such a ""sense-and-avoid"" instrument is desired for autonomous unmanned aerial systems operating in civilian airspace. The instrument was configured as a network of smart camera nodes and implemented using commercial, off-the-shelf components. An end-to-end imaging train model was developed and important figures of merit were derived. Transfer functions arising from intermediate mediums were discussed and their impact assessed. Multiple prototypes were developed. The expected performance of the instrument was iteratively evaluated on the prototypes, beginning with modeling activities followed by laboratory tests, ground tests and flight tests. A prototype was mounted on a Bell 205 helicopter for flight tests, with a Bell 206 helicopter acting as the target. Raw imagery was recorded alongside ancillary aircraft data, and stored for the offline assessment of performance. The ""range at first detection"" (R0), is presented as a robust measure of sensor performance, based on a suitably defined signal-to-noise ratio. The analysis treats target radiance fluctuations, ground clutter, atmospheric effects, platform motion and random noise elements. Under the measurement conditions, R0 exceeded flight crew acquisition ranges. Secondary figures of merit are also discussed, including time to impact, target size and growth, and the impact of resolution on detection range. The hardware was structured to facilitate a real-time hierarchical image-processing pipeline, with selected image processing techniques introduced. In particular, the height of an observed event above the horizon compensates for angular motion of the helicopter platform.

Target detection, tracking, and localization using multi-spectral image fusion and RF Doppler differentials

It is critical for defense and security applications to have a high probability of detection and low false alarm rate while operating over a wide variety of conditions. Sensor fusion, which is the the process of combining data from two or more sensors, has been utilized to improve the performance of a system by exploiting the strengths of each sensor. This dissertation presents algorithms to fuse multi-sensor data that improves system performance by increasing detection rates, lowering false alarms, and improving track performance. Furthermore, this dissertation presents a framework for comparing algorithm error for image registration which is a critical pre-processing step for multi-spectral image fusion. First, I present an algorithm to improve detection and tracking performance for moving targets in a cluttered urban environment by fusing foreground maps from multi-spectral imagery. Most research in image fusion consider visible and long-wave infrared bands; I examine these bands along with near infrared and mid-wave infrared. To localize and track a particular target of interest, I present an algorithm to fuse output from the multi-spectral image tracker with a constellation of RF sensors measuring a specific cellular emanation. The fusion algorithm matches the Doppler differential from the RF sensors with the theoretical Doppler Differential of the video tracker output by selecting the sensor pair that minimizes the absolute difference or root-mean-square difference. Finally, a framework to quantify shift-estimation error for both area- and feature-based algorithms is presented. By exploiting synthetically generated visible and long-wave infrared imagery, error metrics are computed and compared for a number of area- and feature-based shift estimation algorithms. A number of key results are presented in this dissertation. The multi-spectral image tracker improves the location accuracy of the algorithm while improving the detection rate and lowering false alarms for most spectral bands. All 12 moving targets were tracked through the video sequence with only one lost track that was later recovered. Targets from the multi-spectral tracking algorithm were correctly associated with their corresponding cellular emanation for all targets at lower measurement uncertainty using the root-mean-square difference while also having a high confidence ratio for selecting the true target from background targets. For the area-based algorithms and the synthetic air-field image pair, the DFT and ECC algorithms produces sub-pixel shift-estimation error in regions such as shadows and high contrast painted line regions. The edge orientation feature descriptors increase the number of sub-field estimates while improving the shift-estimation error compared to the Lowe descriptor

Modeling Atmosphere-Ocean Radiative Transfer: A PACE Mission Perspective

The research frontiers of radiative transfer (RT) in coupled atmosphere-ocean systems are explored to enable new science and specifically to support the upcoming Plankton, Aerosol, Cloud ocean Ecosystem (PACE) satellite mission. Given (i) the multitude of atmospheric and oceanic constituents at any given moment that each exhibits a large variety of physical and chemical properties and (ii) the diversity of light-matter interactions (scattering, absorption, and emission), tackling all outstanding RT aspects related to interpreting and/or simulating light reflected by atmosphere-ocean systems becomes impossible. Instead, we focus on both theoretical and experimental studies of RT topics important to the science threshold and goal questions of the PACE mission and the measurement capabilities of its instruments. We differentiate between (a) forward (FWD) RT studies that focus mainly on sensitivity to influencing variables and/or simulating data sets, and (b) inverse (INV) RT studies that also involve the retrieval of atmosphere and ocean parameters. Our topics cover (1) the ocean (i.e., water body): absorption and elastic/inelastic scattering by pure water (FWD RT) and models for scattering and absorption by particulates (FWD RT and INV RT); (2) the air-water interface: variations in ocean surface refractive index (INV RT) and in whitecap reflectance (INV RT); (3) the atmosphere: polarimetric and/or hyperspectral remote sensing of aerosols (INV RT) and of gases (FWD RT); and (4) atmosphere-ocean systems: benchmark comparisons, impact of the Earth's sphericity and adjacency effects on space-borne observations, and scattering in the ultraviolet regime (FWD RT). We provide for each topic a summary of past relevant (heritage) work, followed by a discussion (for unresolved questions) and RT updates

Exploring Himawari-8 geostationary observations for the advanced coastal monitoring of the Great Barrier Reef

Larissa developed an algorithm to enable water-quality assessment within the Great Barrier Reef (GBR) using weather satellite observations collected every 10 minutes. This unprecedented temporal resolution records the dynamic nature of water quality fluctuations for the entire GBR, with applications for improved monitoring and management

Mineral identification using data-mining in hyperspectral infrared imagery

Les applications de l’imagerie infrarouge dans le domaine de la géologie sont principalement des applications hyperspectrales. Elles permettent entre autre l’identification minérale, la cartographie, ainsi que l’estimation de la portée. Le plus souvent, ces acquisitions sont réalisées in-situ soit à l’aide de capteurs aéroportés, soit à l’aide de dispositifs portatifs. La découverte de minéraux indicateurs a permis d’améliorer grandement l’exploration minérale. Ceci est en partie dû à l’utilisation d’instruments portatifs. Dans ce contexte le développement de systèmes automatisés permettrait d’augmenter à la fois la qualité de l’exploration et la précision de la détection des indicateurs. C’est dans ce cadre que s’inscrit le travail mené dans ce doctorat. Le sujet consistait en l’utilisation de méthodes d’apprentissage automatique appliquées à l’analyse (au traitement) d’images hyperspectrales prises dans les longueurs d’onde infrarouge. L’objectif recherché étant l’identification de grains minéraux de petites tailles utilisés comme indicateurs minéral -ogiques. Une application potentielle de cette recherche serait le développement d’un outil logiciel d’assistance pour l’analyse des échantillons lors de l’exploration minérale. Les expériences ont été menées en laboratoire dans la gamme relative à l’infrarouge thermique (Long Wave InfraRed, LWIR) de 7.7m à 11.8 m. Ces essais ont permis de proposer une méthode pour calculer l’annulation du continuum. La méthode utilisée lors de ces essais utilise la factorisation matricielle non négative (NMF). En utlisant une factorisation du premier ordre on peut déduire le rayonnement de pénétration, lequel peut ensuite être comparé et analysé par rapport à d’autres méthodes plus communes. L’analyse des résultats spectraux en comparaison avec plusieurs bibliothèques existantes de données a permis de mettre en évidence la suppression du continuum. Les expérience ayant menés à ce résultat ont été conduites en utilisant une plaque Infragold ainsi qu’un objectif macro LWIR. L’identification automatique de grains de différents matériaux tels que la pyrope, l’olivine et le quartz a commencé. Lors d’une phase de comparaison entre des approches supervisées et non supervisées, cette dernière s’est montrée plus approprié en raison du comportement indépendant par rapport à l’étape d’entraînement. Afin de confirmer la qualité de ces résultats quatre expériences ont été menées. Lors d’une première expérience deux algorithmes ont été évalués pour application de regroupements en utilisant l’approche FCC (False Colour Composite). Cet essai a permis d’observer une vitesse de convergence, jusqu’a vingt fois plus rapide, ainsi qu’une efficacité significativement accrue concernant l’identification en comparaison des résultats de la littérature. Cependant des essais effectués sur des données LWIR ont montré un manque de prédiction de la surface du grain lorsque les grains étaient irréguliers avec présence d’agrégats minéraux. La seconde expérience a consisté, en une analyse quantitaive comparative entre deux bases de données de Ground Truth (GT), nommée rigid-GT et observed-GT (rigide-GT: étiquet manuel de la région, observée-GT:étiquetage manuel les pixels). La précision des résultats était 1.5 fois meilleur lorsque l’on a utlisé la base de données observed-GT que rigid-GT. Pour les deux dernières epxérience, des données venant d’un MEB (Microscope Électronique à Balayage) ainsi que d’un microscopie à fluorescence (XRF) ont été ajoutées. Ces données ont permis d’introduire des informations relatives tant aux agrégats minéraux qu’à la surface des grains. Les résultats ont été comparés par des techniques d’identification automatique des minéraux, utilisant ArcGIS. Cette dernière a montré une performance prometteuse quand à l’identification automatique et à aussi été utilisée pour la GT de validation. Dans l’ensemble, les quatre méthodes de cette thèse représentent des méthodologies bénéfiques pour l’identification des minéraux. Ces méthodes présentent l’avantage d’être non-destructives, relativement précises et d’avoir un faible coût en temps calcul ce qui pourrait les qualifier pour être utilisée dans des conditions de laboratoire ou sur le terrain.The geological applications of hyperspectral infrared imagery mainly consist in mineral identification, mapping, airborne or portable instruments, and core logging. Finding the mineral indicators offer considerable benefits in terms of mineralogy and mineral exploration which usually involves application of portable instrument and core logging. Moreover, faster and more mechanized systems development increases the precision of identifying mineral indicators and avoid any possible mis-classification. Therefore, the objective of this thesis was to create a tool to using hyperspectral infrared imagery and process the data through image analysis and machine learning methods to identify small size mineral grains used as mineral indicators. This system would be applied for different circumstances to provide an assistant for geological analysis and mineralogy exploration. The experiments were conducted in laboratory conditions in the long-wave infrared (7.7μm to 11.8μm - LWIR), with a LWIR-macro lens (to improve spatial resolution), an Infragold plate, and a heating source. The process began with a method to calculate the continuum removal. The approach is the application of Non-negative Matrix Factorization (NMF) to extract Rank-1 NMF and estimate the down-welling radiance and then compare it with other conventional methods. The results indicate successful suppression of the continuum from the spectra and enable the spectra to be compared with spectral libraries. Afterwards, to have an automated system, supervised and unsupervised approaches have been tested for identification of pyrope, olivine and quartz grains. The results indicated that the unsupervised approach was more suitable due to independent behavior against training stage. Once these results obtained, two algorithms were tested to create False Color Composites (FCC) applying a clustering approach. The results of this comparison indicate significant computational efficiency (more than 20 times faster) and promising performance for mineral identification. Finally, the reliability of the automated LWIR hyperspectral infrared mineral identification has been tested and the difficulty for identification of the irregular grain’s surface along with the mineral aggregates has been verified. The results were compared to two different Ground Truth(GT) (i.e. rigid-GT and observed-GT) for quantitative calculation. Observed-GT increased the accuracy up to 1.5 times than rigid-GT. The samples were also examined by Micro X-ray Fluorescence (XRF) and Scanning Electron Microscope (SEM) in order to retrieve information for the mineral aggregates and the grain’s surface (biotite, epidote, goethite, diopside, smithsonite, tourmaline, kyanite, scheelite, pyrope, olivine, and quartz). The results of XRF imagery compared with automatic mineral identification techniques, using ArcGIS, and represented a promising performance for automatic identification and have been used for GT validation. In overall, the four methods (i.e. 1.Continuum removal methods; 2. Classification or clustering methods for mineral identification; 3. Two algorithms for clustering of mineral spectra; 4. Reliability verification) in this thesis represent beneficial methodologies to identify minerals. These methods have the advantages to be a non-destructive, relatively accurate and have low computational complexity that might be used to identify and assess mineral grains in the laboratory conditions or in the field