Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

International audienceHyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Non-linear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand, an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that, an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time

Image fusion and spectral unmixing of hyperspectral images for spatial improvement of classification maps

International audienceIn this paper we propose a new approach for the improvement of the spatial resolution of hyperspectral image classification maps combining both spectral unmixing and pansharpening approaches. The main idea is to use a spectral unmixing algorithm based on neural networks to retrieve the abundances of the endmembers present in the scene, and then use the spatial information retrieved from the pansharpened image to find the location of each endmember within the enhanced pixel according to the endmembers abundances. The proposed approach has been applied both to real and synthetic datasets

A convex formulation for hyperspectral image superresolution via subspace-based regularization

Hyperspectral remote sensing images (HSIs) usually have high spectral resolution and low spatial resolution. Conversely, multispectral images (MSIs) usually have low spectral and high spatial resolutions. The problem of inferring images which combine the high spectral and high spatial resolutions of HSIs and MSIs, respectively, is a data fusion problem that has been the focus of recent active research due to the increasing availability of HSIs and MSIs retrieved from the same geographical area. We formulate this problem as the minimization of a convex objective function containing two quadratic data-fitting terms and an edge-preserving regularizer. The data-fitting terms account for blur, different resolutions, and additive noise. The regularizer, a form of vector Total Variation, promotes piecewise-smooth solutions with discontinuities aligned across the hyperspectral bands. The downsampling operator accounting for the different spatial resolutions, the non-quadratic and non-smooth nature of the regularizer, and the very large size of the HSI to be estimated lead to a hard optimization problem. We deal with these difficulties by exploiting the fact that HSIs generally "live" in a low-dimensional subspace and by tailoring the Split Augmented Lagrangian Shrinkage Algorithm (SALSA), which is an instance of the Alternating Direction Method of Multipliers (ADMM), to this optimization problem, by means of a convenient variable splitting. The spatial blur and the spectral linear operators linked, respectively, with the HSI and MSI acquisition processes are also estimated, and we obtain an effective algorithm that outperforms the state-of-the-art, as illustrated in a series of experiments with simulated and real-life data.Comment: IEEE Trans. Geosci. Remote Sens., to be publishe

Spectral transformation based on nonlinear principal component analysis for dimensionality reduction of hyperspectral images

Publisher's version (útgefin grein)Managing transmission and storage of hyperspectral (HS) images can be extremely difficult. Thus, the dimensionality reduction of HS data becomes necessary. Among several dimensionality reduction techniques, transform-based have found to be effective for HS data. While spatial transformation techniques provide good compression rates, the choice of the spectral decorrelation approaches can have strong impact on the quality of the compressed image. Since HS images are highly correlated within each spectral band and in particular across neighboring bands, the choice of a decorrelation method allowing to retain as much information content as possible is desirable. From this point of view, several methods based on PCA and Wavelet have been presented in the literature. In this paper, we propose the use of NLPCA transform as a method to reduce the spectral dimensionality of HS data. NLPCA represents in a lower dimensional space the same information content with less features than PCA. In these terms, aim of this research is focused on the analysis of the results obtained through the spectral decorrelation phase rather than taking advantage of both spectral and spatial compression. Experimental results assessing the advantage of NLPCA with respect to standard PCA are presented on four different HS datasets.This work was supported by the Agence Nationale de la Recherche [project APHYPIS]Peer Reviewe

Bayesian Fusion of Multi-Band Images -Complementary results and supporting materials

Abstract In this paper, a Bayesian fusion technique for remotely sensed multi-band images is presented. The observed images are related to the high spectral and high spatial resolution image to be recovered through physical degradations, e.g., spatial and spectral blurring and/or subsampling defined by the sensor characteristics. The fusion problem is formulated within a Bayesian estimation framework. An appropriate prior distribution exploiting geometrical consideration is introduced. To compute the Bayesian estimator of the scene of interest from its posterior distribution, a Markov chain Monte Carlo algorithm is designed to generate samples asymptotically distributed according to the target distribution. To efficiently sample from this high-dimension distribution, a Hamiltonian Monte Carlo step is introduced in the Gibbs sampling strategy. The efficiency of the proposed fusion method is evaluated with respect to several state-of-the-art fusion techniques. In particular, low spatial resolution hyperspectral and multispectral images are fused to produce a high spatial resolution hyperspectral image. Index Terms Part of this work has been supported by the Hypanema ANR Project

Fusion de données provenant de différents capteurs satellitaires pour le suivi de la qualité de l'eau en zones côtières. Application au littoral de la région PACA

Monitoring coastal areas requires both a good spatial resolution, good spectral resolution associated with agood signal to noise ratio and finally a good temporal resolution to visualize rapid changes in water color.Available now, and even those planed soon, sensors do not provide both a good spatial, spectral ANDtemporal resolution. In this study, we are interested in the image fusion of two future sensors which are bothpart of the Copernicus program of the European Space Agency: MSI on Sentinel-2 and OLCI on Sentinel-3.Such as MSI and OLCI do not provide image yet, it was necessary to simulate them. We then used thehyperspectral imager HICO and we then proposed three methods: an adaptation of the method ARSIS fusionof multispectral images (ARSIS), a fusion method based on the non-negative factorization tensors (Tensor)and a fusion method based on the inversion de matrices (Inversion).These three methods were first evaluated using statistical parameters between images obtained by fusionand the "perfect" image as well as the estimation results of biophysical parameters obtained by minimizingthe radiative transfer model in water.Le suivi des zones côtières nécessite à la fois une bonne résolution spatiale, une bonne résolution spectraleassociée à un bon rapport signal sur bruit et enfin une bonne résolution temporelle pour visualiser deschangements rapides de couleur de l’eau.Les capteurs disponibles actuellement, et même ceux prévus prochainement, n’apportent pas à la fois unebonne résolution spatiale, spectrale ET temporelle. Dans cette étude, nous nous intéressons à la fusion de 2futurs capteurs qui s’inscrivent tous deux dans le programme Copernicus de l’agence spatiale européenne:MSI sur Sentinel-2 et OLCI sur Sentinel-3.Comme les capteurs MSI et OLCI ne fournissent pas encore d’images, il a fallu les simuler. Pour cela nousavons eu recours aux images hyperspectrales du capteur HICO. Nous avons alors proposé 3 méthodes : uneadaptation de la méthode ARSIS à la fusion d’images multispectrales (ARSIS), une méthode de fusion baséesur la factorisation de tenseurs non-négatifs (Tenseur) et une méthode de fusion basée sur l’inversion dematrices (Inversion)Ces 3 méthodes ont tout d’abord été évaluées à l’aide de paramètres statistiques entre les images obtenuespar fusion et l’image « parfaite » ainsi que sur les résultats d’estimation de paramètres biophysiques obtenuspar minimisation du modèle de transfert radiatif dans l’eau

Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction

International audienceThis paper presents a novel method for the enhancement of spatial quality of Hyperspectral (HS) images while making use of a high resolution panchromatic (PAN) image. Due to the high number of bands the application of a pansharpening technique to HS images may result in an increase of the computational load and complexity. Thus a dimensionality reduction preprocess, compressing the original number of measurements into a lower dimensional space, becomes mandatory. To solve this problem we propose a pansharpening technique combining both dimensionality reduction and fusion, exploited by non-linear Principal Component Analysis (NLPCA) and Indusion respectively, to enhance the spatial resolution of a hyperspectral image

Image Fusion in Remote Sensing and Quality Evaluation of Fused Images

In remote sensing, acquired optical images of high spectral resolution have usually a lower spatial resolution than images of lower spectral resolution. This is due to physical, cost and complexity constraints. To make the most of the available imagery, many image fusion techniques have been developed to address this problem. Image fusion is an ill-posed inverse problem where an image of low spatial resolution and high spectral resolution is enhanced in spatial-resolution by using an auxiliary image of high spatial resolution and low spectral resolution. It is assumed that both images display the same scene and are properly co-registered. Thus, the problem is essentially to transfer details from the higher spatial resolution auxiliary image to the upscaled lower resolution image in a manner that minimizes the spatial and spectral distortion of the fused image. The most common image fusion problem is pansharpening, where a multispectral (MS) image is enhanced using wide-band panchromatic (PAN) image. A similar problem is the enhancement of a hyperspectral (HS) image by either a PAN image or an MS image. As there is no reference image available, the reliable quantitative evaluation of the quality of the fused image is a difficult problem. This thesis addresses the image fusion problem in three different ways and also addresses the problem of quantitative quality evaluation.Í fjarkönnun hafa myndir með háa rófsupplausn lægri rúmupplausn en myndir með lægri rófsupplausn vegna eðlisfræðilegra og kostnaðarlegra takmarkana. Til að auka upplýsingamagn slíkra mynda hafa verið þróaðar fjölmargar sambræðsluaðferðir á síðustu tveimur áratugum. Myndsambræðsla er illa framsett andhverft vandmál (e. inverse problem) þar sem rúmupplausn myndar af hárri rófsupplausn er aukin með því að nota upplýsingar frá mynd af hárri rúmupplausn og lægri rófsupplausn. Það er gert ráð fyrir að báðar myndir sýni nákvæmlega sama landsvæði. Þannig er vandamálið í eðli sínu að flytja fíngerða eiginleika myndar af hærri rúmupplausn yfir á mynd af lægri rúmupplausn sem hefur verið brúuð upp í stærð hinnar myndarinnar, án þess að skerða gæði rófsupplýsinga upphaflegu myndarinnar. Algengasta myndbræðsluvandamálið í fjarkönnun er svokölluð panskerpun (e. pansharpening) þar sem fjölrásamynd (e. multispectral image) er endurbætt í rúmi með svokallaðri víðbandsmynd (e. panchromatic image) sem hefur aðeins eina rás af hárri upplausn. Annað svipað vandamál er sambræðsla háfjölrásamyndar (e. hyperspectral image) og annaðhvort fjölrásamyndar eða víðbandsmyndar. Þar sem myndsambræðsla er andhverft vandmál er engin háupplausnar samanburðarmynd tiltæk, sem gerir mat á gæðum sambræddu myndarinnar að erfiðu vandamáli. Í þessari ritgerð eru kynntar þrjár aðferðir sem taka á myndsambræðlsu og einnig er fjallað um mat á gæðum sambræddra mynda, þá sérstaklega panskerptra mynda