Graph-based Data Modeling and Analysis for Data Fusion in Remote Sensing

Hyperspectral imaging provides the capability of increased sensitivity and discrimination over traditional imaging methods by combining standard digital imaging with spectroscopic methods. For each individual pixel in a hyperspectral image (HSI), a continuous spectrum is sampled as the spectral reflectance/radiance signature to facilitate identification of ground cover and surface material. The abundant spectrum knowledge allows all available information from the data to be mined. The superior qualities within hyperspectral imaging allow wide applications such as mineral exploration, agriculture monitoring, and ecological surveillance, etc. The processing of massive high-dimensional HSI datasets is a challenge since many data processing techniques have a computational complexity that grows exponentially with the dimension. Besides, a HSI dataset may contain a limited number of degrees of freedom due to the high correlations between data points and among the spectra. On the other hand, merely taking advantage of the sampled spectrum of individual HSI data point may produce inaccurate results due to the mixed nature of raw HSI data, such as mixed pixels, optical interferences and etc. Fusion strategies are widely adopted in data processing to achieve better performance, especially in the field of classification and clustering. There are mainly three types of fusion strategies, namely low-level data fusion, intermediate-level feature fusion, and high-level decision fusion. Low-level data fusion combines multi-source data that is expected to be complementary or cooperative. Intermediate-level feature fusion aims at selection and combination of features to remove redundant information. Decision level fusion exploits a set of classifiers to provide more accurate results. The fusion strategies have wide applications including HSI data processing. With the fast development of multiple remote sensing modalities, e.g. Very High Resolution (VHR) optical sensors, LiDAR, etc., fusion of multi-source data can in principal produce more detailed information than each single source. On the other hand, besides the abundant spectral information contained in HSI data, features such as texture and shape may be employed to represent data points from a spatial perspective. Furthermore, feature fusion also includes the strategy of removing redundant and noisy features in the dataset. One of the major problems in machine learning and pattern recognition is to develop appropriate representations for complex nonlinear data. In HSI processing, a particular data point is usually described as a vector with coordinates corresponding to the intensities measured in the spectral bands. This vector representation permits the application of linear and nonlinear transformations with linear algebra to find an alternative representation of the data. More generally, HSI is multi-dimensional in nature and the vector representation may lose the contextual correlations. Tensor representation provides a more sophisticated modeling technique and a higher-order generalization to linear subspace analysis. In graph theory, data points can be generalized as nodes with connectivities measured from the proximity of a local neighborhood. The graph-based framework efficiently characterizes the relationships among the data and allows for convenient mathematical manipulation in many applications, such as data clustering, feature extraction, feature selection and data alignment. In this thesis, graph-based approaches applied in the field of multi-source feature and data fusion in remote sensing area are explored. We will mainly investigate the fusion of spatial, spectral and LiDAR information with linear and multilinear algebra under graph-based framework for data clustering and classification problems

Low-Rank and Sparse Decomposition for Hyperspectral Image Enhancement and Clustering

In this dissertation, some new algorithms are developed for hyperspectral imaging analysis enhancement. Tensor data format is applied in hyperspectral dataset sparse and low-rank decomposition, which could enhance the classification and detection performance. And multi-view learning technique is applied in hyperspectral imaging clustering. Furthermore, kernel version of multi-view learning technique has been proposed, which could improve clustering performance. Most of low-rank and sparse decomposition algorithms are based on matrix data format for HSI analysis. As HSI contains high spectral dimensions, tensor based extended low-rank and sparse decomposition (TELRSD) is proposed in this dissertation for better performance of HSI classification with low-rank tensor part, and HSI detection with sparse tensor part. With this tensor based method, HSI is processed in 3D data format, and information between spectral bands and pixels maintain integrated during decomposition process. This proposed algorithm is compared with other state-of-art methods. And the experiment results show that TELRSD has the best performance among all those comparison algorithms. HSI clustering is an unsupervised task, which aims to group pixels into different groups without labeled information. Low-rank sparse subspace clustering (LRSSC) is the most popular algorithms for this clustering task. The spatial-spectral based multi-view low-rank sparse subspace clustering (SSMLC) algorithms is proposed in this dissertation, which extended LRSSC with multi-view learning technique. In this algorithm, spectral and spatial views are created to generate multi-view dataset of HSI, where spectral partition, morphological component analysis (MCA) and principle component analysis (PCA) are applied to create others views. Furthermore, kernel version of SSMLC (k-SSMLC) also has been investigated. The performance of SSMLC and k-SSMLC are compared with sparse subspace clustering (SSC), low-rank sparse subspace clustering (LRSSC), and spectral-spatial sparse subspace clustering (S4C). It has shown that SSMLC could improve the performance of LRSSC, and k-SSMLC has the best performance. The spectral clustering has been proved that it equivalent to non-negative matrix factorization (NMF) problem. In this case, NMF could be applied to the clustering problem. In order to include local and nonlinear features in data source, orthogonal NMF (ONMF), graph-regularized NMF (GNMF) and kernel NMF (k-NMF) has been proposed for better clustering performance. The non-linear orthogonal graph NMF combine both kernel, orthogonal and graph constraints in NMF (k-OGNMF), which push up the clustering performance further. In the HSI domain, kernel multi-view based orthogonal graph NMF (k-MOGNMF) is applied for subspace clustering, where k-OGNMF is extended with multi-view algorithm, and it has better performance and computation efficiency

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas

Inférence bayésienne dans des problèmes inverses, myopes et aveugles en traitement du signal et des images

Les activités de recherche présentées concernent la résolution de problèmes inverses, myopes et aveugles rencontrés en traitement du signal et des images. Les méthodes de résolution privilégiées reposent sur une démarche d'inférence bayésienne. Celle-ci offre un cadre d'étude générique pour régulariser les problèmes généralement mal posés en exploitant les contraintes inhérentes aux modèles d'observation. L'estimation des paramètres d'intérêt est menée à l'aide d'algorithmes de Monte Carlo qui permettent d'explorer l'espace des solutions admissibles. Un des domaines d'application visé par ces travaux est l'imagerie hyperspectrale et, plus spécifiquement, le démélange spectral. Le second travail présenté concerne la reconstruction d'images parcimonieuses acquises par un microscope MRFM

Fusion de données provenant de différents capteurs satellitaires pour le suivi de la qualité de l'eau en zones côtières. Application au littoral de la région PACA

Monitoring coastal areas requires both a good spatial resolution, good spectral resolution associated with agood signal to noise ratio and finally a good temporal resolution to visualize rapid changes in water color.Available now, and even those planed soon, sensors do not provide both a good spatial, spectral ANDtemporal resolution. In this study, we are interested in the image fusion of two future sensors which are bothpart of the Copernicus program of the European Space Agency: MSI on Sentinel-2 and OLCI on Sentinel-3.Such as MSI and OLCI do not provide image yet, it was necessary to simulate them. We then used thehyperspectral imager HICO and we then proposed three methods: an adaptation of the method ARSIS fusionof multispectral images (ARSIS), a fusion method based on the non-negative factorization tensors (Tensor)and a fusion method based on the inversion de matrices (Inversion).These three methods were first evaluated using statistical parameters between images obtained by fusionand the "perfect" image as well as the estimation results of biophysical parameters obtained by minimizingthe radiative transfer model in water.Le suivi des zones côtières nécessite à la fois une bonne résolution spatiale, une bonne résolution spectraleassociée à un bon rapport signal sur bruit et enfin une bonne résolution temporelle pour visualiser deschangements rapides de couleur de l’eau.Les capteurs disponibles actuellement, et même ceux prévus prochainement, n’apportent pas à la fois unebonne résolution spatiale, spectrale ET temporelle. Dans cette étude, nous nous intéressons à la fusion de 2futurs capteurs qui s’inscrivent tous deux dans le programme Copernicus de l’agence spatiale européenne:MSI sur Sentinel-2 et OLCI sur Sentinel-3.Comme les capteurs MSI et OLCI ne fournissent pas encore d’images, il a fallu les simuler. Pour cela nousavons eu recours aux images hyperspectrales du capteur HICO. Nous avons alors proposé 3 méthodes : uneadaptation de la méthode ARSIS à la fusion d’images multispectrales (ARSIS), une méthode de fusion baséesur la factorisation de tenseurs non-négatifs (Tenseur) et une méthode de fusion basée sur l’inversion dematrices (Inversion)Ces 3 méthodes ont tout d’abord été évaluées à l’aide de paramètres statistiques entre les images obtenuespar fusion et l’image « parfaite » ainsi que sur les résultats d’estimation de paramètres biophysiques obtenuspar minimisation du modèle de transfert radiatif dans l’eau

Analyse de séries temporelles d’images à moyenne résolution spatiale : reconstruction de profils de LAI, démélangeage : application pour le suivi de la végétation sur des images MODIS

This PhD dissertation is concerned with time series analysis for medium spatial resolution (MSR) remote sensing images. The main advantage of MSR data is their high temporal rate which allows to monitor land use. However, two main problems arise with such data. First, because of cloud coverage and bad acquisition conditions, the resulting time series are often corrupted and not directly exploitable. Secondly, pixels in medium spatial resolution images are often “mixed” in the sense that the spectral response is a combination of the response of “pure” elements.These two problems are addressed in this PhD. First, we propose a data assimilation technique able to recover consistent time series of Leaf Area Index from corrupted MODIS sequences. To this end, a plant growth model, namely GreenLab, is used as a dynamical constraint. Second, we propose a new and efficient unmixing technique for time series. It is in particular based on the use of “elastic” kernels able to properly compare time series shifted in time or of various lengths.Experimental results are shown both on synthetic and real data and demonstrate the efficiency of the proposed methodologies.Cette thèse s’intéresse à l’analyse de séries temporelles d’images satellites à moyenne résolution spatiale. L’intérêt principal de telles données est leur haute répétitivité qui autorise des analyses de l’usage des sols. Cependant, deux problèmes principaux subsistent avec de telles données. En premier lieu, en raison de la couverture nuageuse, des mauvaises conditions d’acquisition, ..., ces données sont souvent très bruitées. Deuxièmement, les pixels associés à la moyenne résolution spatiale sont souvent “mixtes” dans la mesure où leur réponse spectrale est une combinaison de la réponse de plusieurs éléments “purs”. Ces deux problèmes sont abordés dans cette thèse. Premièrement, nous proposons une technique d’assimilation de données capable de recouvrer des séries temporelles cohérentes de LAI (Leaf Area Index) à partir de séquences d’images MODIS bruitées. Pour cela, le modèle de croissance de plantes GreenLab estutilisé. En second lieu, nous proposons une technique originale de démélangeage, qui s’appuie notamment sur des noyaux “élastiques” capables de gérer les spécificités des séries temporelles (séries de taille différentes, décalées dans le temps, ...)Les résultats expérimentaux, sur des données synthétiques et réelles, montrent de bonnes performances des méthodologies proposées

Méthodes de séparation aveugle de sources pour l'imagerie hyperspectrale : application à la télédétection urbaine et à l'astrophysique

Au cours de cette thèse nous avons développé des méthodes de Séparation Aveugle de Sources (SAS) pour des images hyperspectrales, dans le cadre de deux champs d'application : la télédétection urbaine et l'astrophysique. Dans la première partie de la thèse nous nous sommes intéressés au démélange hyperspectral pour des images urbaines, le but étant de retrouver d'une manière non supervisée les matériaux présents sur la scène en extrayant leurs spectres et leurs proportions. La plupart des méthodes de la littérature sont basées sur un modèle linéaire, qui n'est pas valide en milieu urbain à cause des structures 3D. Une première étape a donc été d'établir un modèle de mélange adapté aux milieux urbains, en partant d'équations physiques basées sur la théorie du transfert radiatif. Le modèle final de forme linéaire quadratique invariant spectralement, ainsi que les possibles hypothèses sur les coefficients de mélange, sont justifiés par les résultats obtenus sur des images simulées réalistes. Nous avons ensuite proposé, pour le démélange, des méthodes de SAS fondées sur la FMN (Factorisation en Matrices Non-négatives). Ces méthodes sont basées sur un calcul de gradient qui tient compte des termes quadratiques. La première méthode utilise un algorithme de gradient à pas fixe, à partir de laquelle une version de Newton a aussi été proposée. La dernière méthode est un algorithme FMN multiplicatif. Les méthodes proposées donnent de meilleures performances qu'une méthode linéaire de la littérature. En astrophysique nous avons développé des méthodes de SAS pour des images de champs denses d'étoiles du spectro-imageur MUSE. A cause de la PSF (Point Spread Function), les informations contenues dans les pixels peuvent résulter des contributions de plusieurs étoiles. C'est là que réside l'intérêt de la SAS : extraire, à partir de ces signaux qui sont des mélanges, les spectres des étoiles qui sont donc nos "sources". Le modèle de mélange est linéaire non invariant spectralement. Nous avons proposé une méthode de SAS basée sur la positivité des données. Cette approche exploite le modèle paramétrique de la FSF (Field Spread Function) de MUSE. La méthode mise en place est itérative et alterne l'estimation des spectres par moindres carrés (avec contraintes de positivité) et estimation des paramètres de la FSF par un algorithme de gradient projeté. La méthode proposée donne de bonnes performances sur des images simulées de MUSE.In this work, we developed Blind Source Separation methods (BSS) for hyperspectral images, concerning two applications : urban remote sensing and astrophysics. The first part of this work concerned spectral unmixing for urban images, with the aim of finding, by an unsupervised method, the materials present in the scene, by extracting their spectra and their proportions. Most existing methods rely on a linear model, which is not valid in urban environments because of 3D structures. Therefore, the first step was to derive a mixing model adapted to urban environments, starting from physical equations based on radiative transfer theory. The derived linear-quadratic model, and possible hypotheses on the mixing coefficients, are justified by results obtained with simulated realistic images. We then proposed, for the unmixing, BSS methods based on NMF (Non-negative Matrix Factorization). These methods are based on gradient computation taking into account the quadratic terms.The first method uses a gradient descent algorithm with a constant step, from which we then derived a Newton version. The last proposed method is a multiplicative NMF algorithm. These methods give better performance than a linear method from the literature. Concerning astrophysics, we developed BSS methods for dense field images of the MUSE instrument. Due to the PSF (Point Spread Function) effect, information contained in the pixels can result from contributions of many stars. Hence, there is a need for BSS, to extract from these signals that are mixtures, the star spectra which are our "sources". The mixing model is linear but spectrally non-invariant. We proposed a BSS method based on positivity. This approach uses the parametric model of MUSE FSF (Field Spread Function). The implemented method is iterative and alternates spectra estimation using least squares (with positivity constraint) and FSF parameter estimation by a projected gradient descent algorithm. The proposed method yields good performance with simulated MUSE images