Dynamical spectral unmixing of multitemporal hyperspectral images

In this paper, we consider the problem of unmixing a time series of hyperspectral images. We propose a dynamical model based on linear mixing processes at each time instant. The spectral signatures and fractional abundances of the pure materials in the scene are seen as latent variables, and assumed to follow a general dynamical structure. Based on a simplified version of this model, we derive an efficient spectral unmixing algorithm to estimate the latent variables by performing alternating minimizations. The performance of the proposed approach is demonstrated on synthetic and real multitemporal hyperspectral images.Comment: 13 pages, 10 figure

Kalman Filtering and Expectation Maximization for Multitemporal Spectral Unmixing

The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications presses for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember representations to account for temporal spectral variability. By representing the multitemporal mixing process using a state-space formulation, we are able to exploit the Bayesian filtering machinery to estimate the endmember variability coefficients. Moreover, by assuming that the temporal variability of the abundances is small over short intervals, an efficient implementation of the expectation maximization (EM) algorithm is employed to estimate the abundances and the other model parameters. Simulation results indicate that the proposed strategy outperforms state-of-the-art multitemporal SU algorithms

Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks

Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupervised frameworks. In this work, we propose an unsupervised MTHU algorithm based on variational recurrent neural networks. First, a stochastic model is proposed to represent both the dynamical evolution of the endmembers and their abundances, as well as the mixing process. Moreover, a new model based on a low-dimensional parametrization is used to represent spatial and temporal endmember variability, significantly reducing the amount of variables to be estimated. We propose to formulate MTHU as a Bayesian inference problem. However, the solution to this problem does not have an analytical solution due to the nonlinearity and non-Gaussianity of the model. Thus, we propose a solution based on deep variational inference, in which the posterior distribution of the estimated abundances and endmembers is represented by using a combination of recurrent neural networks and a physically motivated model. The parameters of the model are learned using stochastic backpropagation. Experimental results show that the proposed method outperforms state of the art MTHU algorithms

Variability of the endmembers in spectral unmixing: recent advances

International audienceEndmember variability has been identified as one of the main limitations of the usual Linear Mixing Model, conventionally used to perform spectral unmixing of hyperspectral data. The topic is currently receiving a lot of attention from the community, and many new algorithms have recently been developed to model this variability and take it into account. In this paper, we review state of the art methods dealing with this problem and classify them into three categories: the algorithms based on endmember bundles, the ones based on computational models, and the ones based on parametric physics-based models. We discuss the advantages and drawbacks of each category of methods and list some open problems and current challenges

Unmixing multitemporal hyperspectral images accounting for smooth and abrupt variations

A classical problem in hyperspectral imaging, referred to as hyperspectral unmixing, consists in estimating spectra associated with each material present in an image and their proportions in each pixel. In practice, illumination variations (e.g., due to declivity or complex interactions with the observed materials) and the possible presence of outliers can result in significant changes in both the shape and the amplitude of the measurements, thus modifying the extracted signatures. In this context, sequences of hyperspectral images are expected to be simultaneously affected by such phenomena when acquired on the same area at different time instants. Thus, we propose a hierarchical Bayesian model to simultaneously account for smooth and abrupt spectral variations affecting a set of multitemporal hyperspectral images to be jointly unmixed. This model assumes that smooth variations can be interpreted as the result of endmember variability, whereas abrupt variations are due to significant changes in the imaged scene (e.g., presence of outliers, additional endmembers, etc.). The parameters of this Bayesian model are estimated using samples generated by a Gibbs sampler according to its posterior. Performance assessment is conducted on synthetic data in comparison with state-of-the-art unmixing methods

Unmixing-based gas plume tracking in LWIR hyperspectral video sequences

International audienceIt is now possible to collect hyperspectral video sequences (HVS) at a near real-time frame rate. The wealth of spectral , spatial and temporal information of those sequences is particularly appealing for chemical gas plume tracking. Existing state-of-the-art methods for such applications however produce only a binary information regarding the position and shape of the gas plume in the HVS. Here, we introduce a novel method relying on spectral unmixing considerations to perform chemical gas plume tracking, which provides information related to the gas plume concentration in addition to its spatial localization. The proposed approach is validated and compared with three state-of-the-art methods on a real HVS

Spatiotemporal dynamics of stress factors in wheat analysed by multisensoral remote sensing and geostatistics

Plant stresses, in particular fungal diseases, basically show a high variability in space and time with respect to their impact on the host. Recent ‘Precision Agriculture’ techniques allow for a spatially and temporally adjusted pest control that might reduce the amount of cost-intensive and ecologically harmful agrochemicals. Conventional stress detection techniques such as random monitoring do not meet demands of such optimally placed management actions. The prerequisite is a profound knowledge about the controlled phenomena as well as their accurate sensor-based detection. Therefore, the present study focused on spatiotemporal dynamics of stress factors in wheat, Europe’s main crop. Primarily, the spatiotemporal characteristics of the fungal diseases, powdery mildew (Blumeria graminis) and leaf rust (Puccinia recondita), were analysed by remote sensing techniques and geo-statistics on leaf and field scale. Basically, there are two different approaches to sensor-based detection of crop stresses: near-range sensors and airborne-/satellite-borne sensors. In order to assess the potential of both approaches, various experiments in field and laboratory were carried out with the use of multiple sensors operated at different scales. Besides the spatial dimension of crop stresses, all studies focussed on the temporal dimension of these phenomena, since this is the key question for an operational use of these techniques. In addition, a comparison between multispectral and hyperspectral data gave an indication of their suitability for this purpose. The results exhibit very high spatiotemporal dynamics for both fungal diseases. However, powdery mildew and leaf rust showed different characteristics, with leaf rust showing a more systematic temporal progress. The physiological behaviours of the phenomena, which are strongly influenced by various environmental factors, define the optimal disease detection date as well as the temporal resolution required for sensor-based disease detection. Due to the high spatiotemporal dynamics of the investigated diseases, a general recommendation of optimal detection periods can not be given, but critical periods are highlighted for each pathogen. The results indicate that multispectral remote sensing data with high spatial resolution shows a high potential for quantifying crop vigour by using spectral mixture analyses. Simulated endmembers for the identification of stressed wheat areas were utilized, whereby promising results could be achieved. However, due to the low spectral resolution of these data, a discrimination of stress factors or early disease detection is not possible. Hyperspectral data was therefore used to point out the potential of early detection of crop diseases, which is a crucial and restrictive factor for Precision Agriculture applications. In a laboratory experiment, leaf rust infections could be detected by hyperspectral data five days after inoculation. In a field experiment with respect to early stress detection, it could be demonstrated that hyperspectral data outperformed multispectral data. High accuracy for the detection of powdery mildew infections in the field was thereby achieved. Due to the fact that typical spatiotemporal characteristics for each pathogen were found, there is a high potential for decision support systems, considering all variables that affect the disease progress. Besides the further analysis of hyperspectral data for disease detection, the development of a decision support system is the subject of the upcoming last period of the Research Training Group 722

Modeling spatial and temporal variabilities in hyperspectral image unmixing

Acquired in hundreds of contiguous spectral bands, hyperspectral (HS) images have received an increasing interest due to the significant spectral information they convey about the materials present in a given scene. However, the limited spatial resolution of hyperspectral sensors implies that the observations are mixtures of multiple signatures corresponding to distinct materials. Hyperspectral unmixing is aimed at identifying the reference spectral signatures composing the data -- referred to as endmembers -- and their relative proportion in each pixel according to a predefined mixture model. In this context, a given material is commonly assumed to be represented by a single spectral signature. This assumption shows a first limitation, since endmembers may vary locally within a single image, or from an image to another due to varying acquisition conditions, such as declivity and possibly complex interactions between the incident light and the observed materials. Unless properly accounted for, spectral variability can have a significant impact on the shape and the amplitude of the acquired signatures, thus inducing possibly significant estimation errors during the unmixing process. A second limitation results from the significant size of HS data, which may preclude the use of batch estimation procedures commonly used in the literature, i.e., techniques exploiting all the available data at once. Such computational considerations notably become prominent to characterize endmember variability in multi-temporal HS (MTHS) images, i.e., sequences of HS images acquired over the same area at different time instants. The main objective of this thesis consists in introducing new models and unmixing procedures to account for spatial and temporal endmember variability. Endmember variability is addressed by considering an explicit variability model reminiscent of the total least squares problem, and later extended to account for time-varying signatures. The variability is first estimated using an unsupervised deterministic optimization procedure based on the Alternating Direction Method of Multipliers (ADMM). Given the sensitivity of this approach to abrupt spectral variations, a robust model formulated within a Bayesian framework is introduced. This formulation enables smooth spectral variations to be described in terms of spectral variability, and abrupt changes in terms of outliers. Finally, the computational restrictions induced by the size of the data is tackled by an online estimation algorithm. This work further investigates an asynchronous distributed estimation procedure to estimate the parameters of the proposed models

Remote-Sensing Detection of Invasive Chinese Tallow (Triadica sebifera) in a Floodplain Environment

Chinese tallow (Tradica sebifera) is an established invasive species in many southern woodlands in the United States. Its ability to adapt and spread quickly into disturbed areas has made it an invasive species of much concern to land managers. Riparian/floodplain environments have been affected by tallow as much as upland areas and entail a high degree of Chinese tallow invasion. Remote sensing is a tool that may provide a means of detecting, or classifying, Chinese tallow. There have been very few studies that have attempted to map Chinese tallow in a floodplain environment. This research focused on mapping Chinese tallow on a single river meander bend. The purpose of this study was to determine which of the nonparametric detection methods considered, such as Multivariate Regression Splines (MARS), Stochastic Gradient Boosting (SGB) and the Random Forest (RF) models, as well as common spectral-extraction algorithms, were able to most accurately detect Chinese tallow in a floodplain forest based on remote-sensing data. In addition, it was the purpose of this study to attempt to determine factors affecting tallow growth and spread, and to map the spatial distribution of tallow in the study area. Fieldwork was conducted in 2010 and 2014 to acquire Chinese tallow presence/absence information to be used for classification model training and testing. A hyperspectral Hyperion satellite image from summer 2010 constituted the primary remote-sensing data source, as well as airborne LiDAR data. The three nonparametric models tested were used to predict Chinese tallow occurrences in the study area. A variety of input variables were employed in the modeling process, including: Hyperion image bands, dimensionality-reduced Minimum Noise Fraction (MNF) images, vegetation indices, and topographic and soil variables. An endmember-based approach was also used to classify tallow presence but was not very successful. Results show that the most accurate dataset-combination trials involving both SGB and MARS yield high overall classification accuracy, 92.85%, whereas the most accurate RF dataset-combination trial provides lower overall classification accuracy, at 80%. Both spatial and aspatial statistical analyses were performed on the classification results. Significance testing indicates that the most accurate RF classification is not statistically significantly different from the most accurate SGB and MARS classifications. However, other error matrix significance testing finds the most accurate RF classification to be statistically significantly different from the most accurate SGB and MARS Chinese tallow classifications. A hot-spot analysis revealed that homogenous areas classified as tallow or as non-tallow can be detected and identified. Results from this study are promising in many areas of the meander bend, such as the transition zone where tallow is prevalent but less so in areas that have more established forest. Some methods tested were successful in detecting tallow and their use may aid land managers in the managing Chinese tallow growth and spread