Hyperspectral pan-sharpening: a variational convex constrained formulation to impose parallel level lines, solved with ADMM

In this paper, we address the issue of hyperspectral pan-sharpening, which consists in fusing a (low spatial resolution) hyperspectral image HX and a (high spatial resolution) panchromatic image P to obtain a high spatial resolution hyperspectral image. The problem is addressed under a variational convex constrained formulation. The objective favors high resolution spectral bands with level lines parallel to those of the panchromatic image. This term is balanced with a total variation term as regularizer. Fit-to-P data and fit-to-HX data constraints are effectively considered as mathematical constraints, which depend on the statistics of the data noise measurements. The developed Alternating Direction Method of Multipliers (ADMM) optimization scheme enables us to solve this problem efficiently despite the non differentiabilities and the huge number of unknowns.Comment: 4 pages, detailed version of proceedings of conference IEEE WHISPERS 201

Multiresolution based, multisensor, multispectral image fusion

Spaceborne sensors, which collect imagery of the Earth in various spectral bands, are limited by the data transmission rates. As a result the multispectral bands are transmitted at a lower resolution and only the panchromatic band is transmitted at its full resolution. The information contained in the multispectral bands is an invaluable tool for land use mapping, urban feature extraction, etc. However, the limited spatial resolution reduces the appeal and value of this information. Pan sharpening techniques enhance the spatial resolution of the multispectral imagery by extracting the high spatial resolution of the panchromatic band and adding it to the multispectral images. There are many different pan sharpening methods available like the ones based on the Intensity-Hue-Saturation and the Principal Components Analysis transformation. But these methods cause heavy spectral distortion of the multispectral images. This is a drawback if the pan sharpened images are to be used for classification based applications. In recent years, multiresolution based techniques have received a lot of attention since they preserve the spectral fidelity in the pan sharpened images. Many variations of the multiresolution based techniques exist. They differ based on the transform used to extract the high spatial resolution information from the images and the rules used to synthesize the pan sharpened image. The superiority of many of the techniques has been demonstrated by comparing them with fairly simple techniques like the Intensity-Hue-Saturation or the Principal Components Analysis. Therefore there is much uncertainty in the pan sharpening community as to which technique is the best at preserving the spectral fidelity. This research investigates these variations in order to find an answer to this question. An important parameter of the multiresolution based methods is the number of decomposition levels to be applied. It is found that the number of decomposition levels affects both the spatial and spectral quality of the pan sharpened images. The minimum number of decomposition levels required to fuse the multispectral and panchromatic images was determined in this study for image pairs with different resolution ratios and recommendations are made accordingly

Multiresolution based, multisensor, multispectral image fusion

Spaceborne sensors, which collect imagery of the Earth in various spectral bands, are limited by the data transmission rates. As a result the multispectral bands are transmitted at a lower resolution and only the panchromatic band is transmitted at its full resolution. The information contained in the multispectral bands is an invaluable tool for land use mapping, urban feature extraction, etc. However, the limited spatial resolution reduces the appeal and value of this information. Pan sharpening techniques enhance the spatial resolution of the multispectral imagery by extracting the high spatial resolution of the panchromatic band and adding it to the multispectral images. There are many different pan sharpening methods available like the ones based on the Intensity-Hue-Saturation and the Principal Components Analysis transformation. But these methods cause heavy spectral distortion of the multispectral images. This is a drawback if the pan sharpened images are to be used for classification based applications. In recent years, multiresolution based techniques have received a lot of attention since they preserve the spectral fidelity in the pan sharpened images. Many variations of the multiresolution based techniques exist. They differ based on the transform used to extract the high spatial resolution information from the images and the rules used to synthesize the pan sharpened image. The superiority of many of the techniques has been demonstrated by comparing them with fairly simple techniques like the Intensity-Hue-Saturation or the Principal Components Analysis. Therefore there is much uncertainty in the pan sharpening community as to which technique is the best at preserving the spectral fidelity. This research investigates these variations in order to find an answer to this question. An important parameter of the multiresolution based methods is the number of decomposition levels to be applied. It is found that the number of decomposition levels affects both the spatial and spectral quality of the pan sharpened images. The minimum number of decomposition levels required to fuse the multispectral and panchromatic images was determined in this study for image pairs with different resolution ratios and recommendations are made accordingly

Target-adaptive CNN-based pansharpening

We recently proposed a convolutional neural network (CNN) for remote sensing image pansharpening obtaining a significant performance gain over the state of the art. In this paper, we explore a number of architectural and training variations to this baseline, achieving further performance gains with a lightweight network which trains very fast. Leveraging on this latter property, we propose a target-adaptive usage modality which ensures a very good performance also in the presence of a mismatch w.r.t. the training set, and even across different sensors. The proposed method, published online as an off-the-shelf software tool, allows users to perform fast and high-quality CNN-based pansharpening of their own target images on general-purpose hardware