A manifold learning approach to target detection in high-resolution hyperspectral imagery

Imagery collected from airborne platforms and satellites provide an important medium for remotely analyzing the content in a scene. In particular, the ability to detect a specific material within a scene is of high importance to both civilian and defense applications. This may include identifying targets such as vehicles, buildings, or boats. Sensors that process hyperspectral images provide the high-dimensional spectral information necessary to perform such analyses. However, for a d-dimensional hyperspectral image, it is typical for the data to inherently occupy an m-dimensional space, with m \u3c\u3c d. In the remote sensing community, this has led to a recent increase in the use of manifold learning, which aims to characterize the embedded lower-dimensional, non-linear manifold upon which the hyperspectral data inherently lie. Classic hyperspectral data models include statistical, linear subspace, and linear mixture models, but these can place restrictive assumptions on the distribution of the data; this is particularly true when implementing traditional target detection approaches, and the limitations of these models are well-documented. With manifold learning based approaches, the only assumption is that the data reside on an underlying manifold that can be discretely modeled by a graph. The research presented here focuses on the use of graph theory and manifold learning in hyperspectral imagery. Early work explored various graph-building techniques with application to the background model of the Topological Anomaly Detection (TAD) algorithm, which is a graph theory based approach to anomaly detection. This led towards a focus on target detection, and in the development of a specific graph-based model of the data and subsequent dimensionality reduction using manifold learning. An adaptive graph is built on the data, and then used to implement an adaptive version of locally linear embedding (LLE). We artificially induce a target manifold and incorporate it into the adaptive LLE transformation; the artificial target manifold helps to guide the separation of the target data from the background data in the new, lower-dimensional manifold coordinates. Then, target detection is performed in the manifold space

Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing

Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these hyperspectral (HS) products mainly by means of seasoned experts. However, with the ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges on reducing the burden of manual labor and improving efficiency. For this reason, it is, therefore, urgent to develop more intelligent and automatic approaches for various HS RS applications. Machine learning (ML) tools with convex optimization have successfully undertaken the tasks of numerous artificial intelligence (AI)-related applications. However, their ability in handling complex practical problems remains limited, particularly for HS data, due to the effects of various spectral variabilities in the process of HS imaging and the complexity and redundancy of higher dimensional HS signals. Compared to the convex models, non-convex modeling, which is capable of characterizing more complex real scenes and providing the model interpretability technically and theoretically, has been proven to be a feasible solution to reduce the gap between challenging HS vision tasks and currently advanced intelligent data processing models

Cloud removal from optical remote sensing images

Optical remote sensing images used for Earth surface observations are constantly contaminated by cloud cover. Clouds dynamically affect the applications of optical data and increase the difficulty of image analysis. Therefore, cloud is considered as one of the sources of noise in optical image data, and its detection and removal need to be operated as a pre-processing step in most remote sensing image processing applications. This thesis investigates the current cloud detection and removal algorithms and develops three new cloud removal methods to improve the accuracy of the results. A thin cloud removal method based on signal transmission principles and spectral mixture analysis (ST-SMA) for pixel correction is developed in the first contribution. This method considers not only the additive reflectance from the clouds but also the energy absorption when solar radiation passes through them. Data correction is achieved by subtracting the product of the cloud endmember signature and the cloud abundance and rescaling according to the cloud thickness. The proposed method has no requirement for meteorological data and does not rely on reference images. The experimental results indicate that the proposed approach is able to perform effective removal of thin clouds in different scenarios. In the second study, an effective cloud removal method is proposed by taking advantage of the noise-adjusted principal components transform (CR-NAPCT). It is found that the signal-to-noise ratio (S/N) of cloud data is higher than data without cloud contamination, when spatial correlation is considered and are shown in the first NAPCT component (NAPC1) in the NAPCT data. An inverse transformation with a modified first component is then applied to generate the cloud free image. The effectiveness of the proposed method is assessed by performing experiments on simulated and real data to compare the quantitative and qualitative performance of the proposed approach. The third study of this thesis deals with both cloud and cloud shadow problems with the aid of an auxiliary image in a clear sky condition. A new cloud removal approach called multitemporal dictionary learning (MDL) is proposed. Dictionaries of the cloudy areas (target data) and the cloud free areas (reference data) are learned separately in the spectral domain. An online dictionary learning method is then applied to obtain the two dictionaries in this method. The removal process is conducted by using the coefficients from the reference image and the dictionary learned from the target image. This method is able to recover the data contaminated by thin and thick clouds or cloud shadows. The experimental results show that the MDL method is effective from both quantitative and qualitative viewpoints

Water ice in the dark dune spots of Richardson crater on Mars

In this study we assess the presence, nature and properties of ices - in particular water ice - that occur within these spots using HIRISE and CRISM observations, as well as the LMD Global Climate Model. Our studies focus on Richardson crater (72{\deg}S, 179{\deg}E) and cover southern spring and summer (LS 175{\deg} - 17 341{\deg}). Three units have been identified of these spots: dark core, gray ring and bright halo. Each unit show characteristic changes as the season progress. In winter, the whole area is covered by CO2 ice with H2O ice contamination. Dark spots form during late winter and early spring. During spring, the dark spots are located in a 10 cm thick depression compared to the surrounding bright ice-rich layer. They are spectrally characterized by weak CO2 ice signatures that probably result from spatial mixing of CO2 ice rich and ice free regions within pixels, and from mixing of surface signatures due to aerosols scattering. The bright halo shaped by winds shows stronger CO2 absorptions than the average ice covered terrain, which is consistent with a formation process involving CO2 re-condensation. According to spectral, morphological and modeling considerations, the gray ring is composed of a thin layer of a few tens of {\mu}m of water ice. Two sources/processes could participate to the enrichment of water ice in the gray ring unit: (i) water ice condensation at the surface in early fall (prior to the condensation of a CO2 rich winter layer) or during winter time (due to cold trapping of the CO2 layer); (ii) ejection of dust grains surrounded by water ice by the geyser activity responsible for the dark spot. In any case, water ice remains longer in the gray ring unit after the complete sublimation of the CO2. Finally, we also looked for liquid water in the near-IR CRISM spectra using linear unmixing modeling but found no conclusive evidence for it

A Developed Algorithm for Automating the Multiple Bands Multiple Endmember Selection of Hyperion data Applied on Central of Cairo, Egypt

This study attempts to provide an answer regarding the utility of Hyperion imagery in mapping urban settings in developed countries. The authors present a novel method for extracting quantitative land cover information at the sub-pixel level from hyperspectral or Hyperion imagery. The proposed method is based on the multiple endmember spectral mixture (MESMA) proposed by Roberts et al. (1998b), but extends it to handle the high-dimensional pixels characterizing hyperspectral images. The proposed method utilizes a multiband multiple endmember spectral mixture analysis (Multiband MESMA) model that allows for both spectral bands and endmembers to vary on a per-pixel basis across a hyperspectral image. The goal is to select an optimal subset of spectral bands that maximizes spectral separability among a candidate set of endmembers for a given pixel, and accordingly to minimize spectral confusion among modeled endmembers and increase the accuracy and physical representativeness of derived fractions for that pixel. The authors develop a tool to automate this method and test its utility in a case study using a Hyperion image of Central Cairo, Egypt. The EO-1 Hyperion hyperspectral sensor is the only source of hyperspectral data currently available for Cairo, unlike cities in Europe and North America, where multiple sources of such data generally exist. The study scene represents a very heterogeneous landscape and has an ecological footprint of a complex range of interrelated socioeconomic, environmental and urban dynamics. The results of this study show that Hyperion data, with its rich spectral information, can help address some of the limitations in automated mapping that are reported by previous studies. For this, proper bands and endmembers are selected and used within a multiple endmember, with a multiple-band SMA process to determine the best Root Mean Square Error (RMSE) and abundance percentages. This results in a better mapping of land cover extricated from hyperspectral imagery (Hyperion). Keywords: Spectral Mixture Analysis, Hyperspectral Data, Hyperion Data, Cairo, Egyp

Detection of Marine Plastic Debris in the North Pacific Ocean using Optical Satellite Imagery

Plastic pollution is ubiquitous across marine environments, yet detection of anthropogenic debris in the global oceans is in its infancy. Here, we exploit high-resolution multispectral satellite imagery over the North Pacific Ocean and information from GPS-tracked floating plastic conglomerates to explore the potential for detecting marine plastic debris via spaceborne remote sensing platforms. Through an innovative method of estimating material abundance in mixed pixels, combined with an inverse spectral unmixing calculation, a spectral signature of aggregated plastic litter was derived from an 8-band WorldView-2 image. By leveraging the spectral characteristics of marine plastic debris in a real environment, plastic detectability was demonstrated and evaluated utilising a Spectral Angle Mapper (SAM) classification, Mixture Tuned Matched Filtering (MTMF), the Reed-Xiaoli Detector (RXD) algorithm, and spectral indices in a three-variable feature space. Results indicate that floating aggregations are detectable on sub-pixel scales, but as reliable ground truth information was restricted to a single confirmed target, detections were only validated by means of their respective spectral responses. Effects of atmospheric correction algorithms were evaluated using ACOLITE, ACOMP, and FLAASH, in which derived unbiased percentage differences ranged from 1% to 81% following a pairwise comparison. Building first steps towards an integrated marine monitoring system, the strengths and limitations of current remote sensing technology are identified and adopted to make suggestions for future improvements

Multiscale Imaging of Evapotranspiration

Evapotranspiration (ET; evaporation + transpiration) is central to a wide range of biological, chemical, and physical processes in the Earth system. Accurate remote sensing of ET is challenging due to the interrelated and generally scale dependent nature of the physical factors which contribute to the process. The evaporation of water from porous media like sands and soils is an important subset of the complete ET problem. Chapter 1 presents a laboratory investigation into this question, examining the effects of grain size and composition on the evolution of drying sands. The effects of composition are found to be 2-5x greater than the effects of grain size, indicating that differences in heating caused by differences in reflectance may dominate hydrologic differences caused by grain size variation. In order to relate the results of Chapter 1 to the satellite image archive, however, the question of information loss between hyperspectral (measurements at 100s of wavelength intervals) laboratory measurements and multispectral (≤ 12 wavelength intervals) satellite images must be addressed. Chapter 2 focuses on this question as applied to substrate materials such as sediment, soil, rock, and non-photosynthetic vegetation. The results indicate that the continuum that is resolved by multispectral sensors is sufficient to resolve the gradient between sand-rich and clay-rich soils, and that this gradient is also a dominant feature in hyperspectral mixing spaces where the actual absorptions can be resolved. Multispectral measurements can be converted to biogeophysically relevant quantities using spectral mixture analysis (SMA). However, retrospective multitemporal analysis first requires cross-sensor calibration of the mixture model. Chapter 3 presents this calibration, allowing multispectral image data to be used interchangeably throughout the Landsat 4-8 archive. In addition, a theoretical explanation is advanced for the observed superior scaling properties of SMA-derived fraction images over spectral indices. The physical quantities estimated by the spectral mixture model are then compared to simultaneously imaged surface temperature, as well as to the derived parameters of ET Fraction and Moisture Availability. SMA-derived vegetation abundance is found to produce substantially more informative ET maps, and SMA-derived substrate fraction is found to yield a surprisingly strong linear relationship with surface temperature. These results provide context for agricultural applications. Chapter 5 investigates the question of mapping and monitoring rice agricultural using optical and thermal satellite image time series. Thermal image time series are found to produce more accurate maps of rice presence/absence, but optical image time series are found to produce more accurate maps of rice crop timing. Chapter 6 takes a more global approach, investigating the spatial structure of agricultural networks for a diverse set of landscapes. Surprisingly consistent scaling relations are found. These relations are assessed in the context of a network-based approach to land cover analysis, with potential implications for the scale dependence of ET estimates. In sum, this thesis present a novel approach to improving ET estimation based on a synthesis of complementary laboratory measurements, satellite image analysis, and field observations. Alone, each of these independent sources of information provides novel insights. Viewed together, these insights form the basis of a more accurate and complete geophysical understanding of the ET phenomenon

Monitoring the coastal zone using earth observation::application of linear spectral unmixing to coastal dune systems in Wales

Coastal sand dune systems across temperate Europe are presently characterized by a high level of ecological stabilization and a subsequent loss of biological diversity. The use of continuous monitoring within these systems is vital to the preservation of species richness, particularly with regard to the persistence of early stage pioneer species dependent on a strong sediment supply. Linear spectral unmixing was applied to archived Landsat data (1975?2014) and historical aerial photography (1941?1962) for monitoring bare sand (BS) cover dynamics as a proxy for ecological dune stabilization. Using this approach, a time series of change was calculated for Kenfig Burrows, a 6-km2 stabilized dune system in South Wales, during 1941?2014. The time series indicated that a rapid level of stabilization had occurred within the study area over a period of 75 years. Accuracy assessment of the data indicated the suitability of medium-resolution imagery with an RMSE of <10% across all images and a difference of <3% between observed and predicted BS area. Temporal resolution was found to be a significant factor in the representation of BS cover with fluctuations occurring on a sub-decadal scale, outside of the margin of error introduced through the use of medium-resolution Landsat imagery. This study demonstrates a tractable approach for mapping and monitoring ecologically sensitive regions at a subLandsat pixel levelpublishersversionPeer reviewe