Automated Synthetic Scene Generation

First principles, physics-based models help organizations developing new remote sensing instruments anticipate sensor performance by enabling the ability to create synthetic imagery for proposed sensor before a sensor is built. One of the largest challenges in modeling realistic synthetic imagery, however, is generating the spectrally attributed, three-dimensional scenes on which the models are based in a timely and affordable fashion. Additionally, manual and semi-automated approaches to synthetic scene construction which rely on spectral libraries may not adequately capture the spectral variability of real-world sites especially when the libraries consist of measurements made in other locations or in a lab. This dissertation presents a method to fully automate the generation of synthetic scenes when coincident lidar, Hyperspectral Imagery (HSI), and high-resolution imagery of a real-world site are available. The method, called the Lidar/HSI Direct (LHD) method, greatly reduces the time and manpower needed to generate a synthetic scene while also matching the modeled scene as closely as possible to a real-world site both spatially and spectrally. Furthermore, the LHD method enables the generation of synthetic scenes over sites in which ground access is not available providing the potential for improved military mission planning and increased ability to fuse information from multiple modalities and look angles. The LHD method quickly and accurately generates three-dimensional scenes providing the community with a tool to expand the library of synthetic scenes and therefore expand the potential applications of physics-based synthetic imagery modeling

Realistic texture in simulated thermal infrared imagery

Creating a visually-realistic yet radiometrically-accurate simulation of thermal infrared (TIR) imagery is a challenge that has plagued members of industry and academia alike. The goal of imagery simulation is to provide a practical alternative to the often staggering effort required to collect actual data. Previous attempts at simulating TIR imagery have suffered from a lack of texture—the simulated scenes generally failed to reproduce the natural variability seen in actual TIR images. Realistic synthetic TIR imagery requires modeling sources of variability including surface effects such as solar insolation and convective heat exchange as well as sub-surface effects such as density and water content. This research effort utilized the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model, developed at the Rochester Institute of Technology, to investigate how these additional sources of variability could be modeled to correctly and accurately provide simulated TIR imagery. Actual thermal data were collected, analyzed, and exploited to determine the underlying thermodynamic phenomena and ascertain how these phenomena are best modeled. The underlying task was to determine how to apply texture in the thermal region to attain radiometrically-correct, visually-appealing simulated imagery. Three natural desert scenes were used to test the methodologies that were developed for estimating per-pixel thermal parameters which could then be used for TIR image simulation by DIRSIG. Additional metrics were devised and applied to the synthetic images to further quantify the success of this research. The resulting imagery demonstrated that these new methodologies for modeling TIR phenomena and the utilization of an improved DIRSIG tool improved the root mean-squared error (RMSE) of our synthetic TIR imagery by up to 88%

Assessing and Enabling Independent Component Analysis As A Hyperspectral Unmixing Approach

As a result of its capacity for material discrimination, hyperspectral imaging has been utilized for applications ranging from mining to agriculture to planetary exploration. One of the most common methods of exploiting hyperspectral images is spectral unmixing, which is used to discriminate and locate the various types of materials that are present in the scene. When this processing is done without the aid of a reference library of material spectra, the problem is called blind or unsupervised spectral unmixing. Independent component analysis (ICA) is a blind source separation approach that operates by finding outputs, called independent components, that are statistically independent. ICA has been applied to the unsupervised spectral unmixing problem, producing intriguing, if somewhat unsatisfying results. This dissatisfaction stems from the fact that independent components are subject to a scale ambiguity which must be resolved before they can be used effectively in the context of the spectral unmixing problem. In this dissertation, ICA is explored as a spectral unmixing approach. Various processing steps that are common in many ICA algorithms are examined to assess their impact on spectral unmixing results. Synthetically-generated but physically-realistic data are used to allow the assessment to be quantitative rather than qualitative only. Additionally, two algorithms, class-based abundance rescaling (CBAR) and extended class-based abundance rescaling (CBAR-X), are introduced to enable accurate rescaling of independent components. Experimental results demonstrate the improved rescaling accuracy provided by the CBAR and CBAR-X algorithms, as well as the general viability of ICA as a spectral unmixing approach

Remote Sensing of the Aquatic Environments

The book highlights recent research efforts in the monitoring of aquatic districts with remote sensing observations and proximal sensing technology integrated with laboratory measurements. Optical satellite imagery gathered at spatial resolutions down to few meters has been used for quantitative estimations of harmful algal bloom extent and Chl-a mapping, as well as winds and currents from SAR acquisitions. The knowledge and understanding gained from this book can be used for the sustainable management of bodies of water across our planet

Methods for Generating High-Fidelity Trace Chemical Residue Reflectance Signatures for Active Spectroscopy Classification Applications

Standoff detection and identification of trace chemicals in hyperspectral infrared images is an enabling capability in a variety of applications relevant to defense, law enforcement, and intelligence communities. Performance of these methods is impacted by the spectral signature variability due to the presence of contaminants, surface roughness, nonlinear effects, etc. Though multiple classes of algorithms exist for the detection and classification of these signatures, they are limited by the availability of relevant reference datasets. In this work, we first address the lack of physics-based models that can accurately predict trace chemical spectra. Most available models assume that the chemical takes the form of spherical particles or uniform thin films. A more realistic chemical presentation that could be encountered is that of a non-uniform chemical film that is deposited after evaporation of the solvent which contained the chemical. This research presents an improved signature model for this type of solid film. The proposed model, called sparse transfer matrix (STM), includes a log-normal distribution of film thicknesses and is found to reduce the root-mean-square error between simulated and measured data by about 25% when compared with either the particle or uniform thin film models. When applied to measured data, the sparse transfer matrix model provides a 0.10-0.28 increase in classification accuracy over traditional models. There remain limitations in the STM model which prevent the predicted spectra from being well-matched to the measured data in some cases. To overcome this, we leverage the field of domain adaptation to translate data from the simulated to the measured data domain. This thesis presents the first one-dimensional (1D) conditional generative adversarial network (GAN) to perform spectrum-to-spectrum translation of reflectance signatures. We apply the 1D conditional GAN to a library of simulated spectra and quantify the improvement with the translated library. The method demonstrates an increase in overall classification accuracy to 0.723 from the accuracy of 0.622 achieved using the STM model when tested on real data. However, the performance improvement is biased towards data included in the GAN training set. The next phase of the research focuses on learning models that are more robust to different parameter combinations for which we do not have measured data. This part of the research leverages elements from the field of theory-guided data science. Specifically, we develop a physics-guided neural network (PGNN) for predicting chemical reflectance for a set of parameterized inputs that is more accurate than the state-of-the-art physics-based signature model for chemical residues. After training the PGNN, we use it to generate a library of predicted spectra for training a classifier. We compare the classification accuracy when using this PGNN library versus a library generated by the physics-based model. Using the PGNN, the average classification accuracy increases to 0.813 on real chemical reflectance data, including data from chemicals not included in the PGNN training set. The products of this thesis work include methods for producing realistic trace chemical residue reflectance signatures as well as demonstrations of improved performance in active spectroscopy classification applications. These methods provide great value to a range of scientific communities. The novel STM signature model enables existing spectroscopy sensors and algorithms to perform well on real-world problems where chemical contaminants are non-uniform. The 1D conditional GAN is the first of its kind and can be applied to many other 1D datasets, such as audio and other time-series data. Finally, the application of theory-guided data science to the trace chemical problem not only enhances the quality of results for known targets and backgrounds, but also increases the robustness to new targets

An Evaluation of Trend and Anomalies of Arctic Sea Ice Concentration, 1979-2006

As a part of the Cyrosphere ecosystem, Arctic sea ice is one of the focal points when studying Arctic climate change. Arctic sea ice image has been documented by remotely sensed data since the 1970s. By examining these data, some climate patterns can be revealed. In this research, Arctic region is divided into 9 sections to analyze the regional differences of the ice coverage and variability. Data used are bootstrapped 1979 to 2006 SSM/I and SMMR images from NSIDC to perform a time series analysis to examine the sea ice trends and spatial/temporal anomalies detection by conducting a descending sort of sea ice coverage by years in the sub-regional scale. Then, the temporal mixture analysis developed by Piwowar & LeDrew is applied to the data to reveal the variability within each subregion. Fractional images produced by TMA highlight the temporal signature concentration in the entire Arctic region. And the color-mix image derived from TMA highlights and overlaps temporal signatures that have over 80% concentrations from highest to lowest. The color mix image can reveal the spatial distribution of similar temporal characteristics and the evolution of time series in the same area during the 30-year period. Through this analysis, the spatial and temporal variability of Arctic sea ice can be perceived that in the subpolar regions, Arctic sea ice has a higher seasonal pattern which varies a lot each other. The Arctic sea ice extent endures an overall decline trend, which the decline speed increases every ten years. But this trend is not statistically significant in every subregion. The spatial/temporal anomaly analysis reveals several patterns of Arctic sea ice variability. The seasonal variability of Arctic sea ice in the eastern and western side of the Arctic Basin resemble each other in the long term, which may coincide with the North Atlantic Oscillation. In addition, within a subregion, different areas may have significantly different temporal characteristics, such as the Greenland Sea and Seas of Okhotsk. Moreover, the temporal characteristics some areas in the Arctic region have changed through time significantly regarding early melt or late freeze. Hopefully this analysis will provide undiscovered temporal evolution through time and some new insights on the dynamics of the Arctic sea ice cover

Automated Remote Sensing Image Interpretation with Limited Labeled Training Data

Automated remote sensing image interpretation has been investigated for more than a decade. In early years, most work was based on the assumption that there are sufficient labeled samples to be used for training. However, ground-truth collection is a very tedious and time-consuming task and sometimes very expensive, especially in the field of remote sensing that usually relies on field surveys to collect ground truth. In recent years, as the development of advanced machine learning techniques, remote sensing image interpretation with limited ground-truth has caught the attention of researchers in the fields of both remote sensing and computer science. Three approaches that focus on different aspects of the interpretation process, i.e., feature extraction, classification, and segmentation, are proposed to deal with the limited ground truth problem. First, feature extraction techniques, which usually serve as a pre-processing step for remote sensing image classification are explored. Instead of only focusing on feature extraction, a joint feature extraction and classification framework is proposed based on ensemble local manifold learning. Second, classifiers in the case of limited labeled training data are investigated, and an enhanced ensemble learning method that outperforms state-of-the-art classification methods is proposed. Third, image segmentation techniques are investigated, with the aid of unlabeled samples and spatial information. A semi-supervised self-training method is proposed, which is capable of expanding the number of training samples by its own and hence improving classification performance iteratively. Experiments show that the proposed approaches outperform state-of-the-art techniques in terms of classification accuracy on benchmark remote sensing datasets.4 month