Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

Hyperspectral imaging can help better understand the characteristics of different materials, compared with traditional image systems. However, only high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS) images can generally be captured at video rate in practice. In this paper, we propose a model-based deep learning approach for merging an HrMS and LrHS images to generate a high-resolution hyperspectral (HrHS) image. In specific, we construct a novel MS/HS fusion model which takes the observation models of low-resolution images and the low-rankness knowledge along the spectral mode of HrHS image into consideration. Then we design an iterative algorithm to solve the model by exploiting the proximal gradient method. And then, by unfolding the designed algorithm, we construct a deep network, called MS/HS Fusion Net, with learning the proximal operators and model parameters by convolutional neural networks. Experimental results on simulated and real data substantiate the superiority of our method both visually and quantitatively as compared with state-of-the-art methods along this line of research.Comment: 10 pages, 7 figure

Detection of leaf structures in close-range hyperspectral images using morphological fusion

Close-range hyperspectral images are a promising source of information in plant biology, in particular, for in vivo study of physiological changes. In this study, we investigate how data fusion can improve the detection of leaf elements by combining pixel reflectance and morphological information. The detection of image regions associated to the leaf structures is the first step toward quantitative analysis on the physical effects that genetic manipulation, disease infections, and environmental conditions have in plants. We tested our fusion approach on Musa acuminata (banana) leaf images and compared its discriminant capability to similar techniques used in remote sensing. Experimental results demonstrate the efficiency of our fusion approach, with significant improvements over some conventional methods

Single-image super-resolution of sentinel-2 low resolution bands with residual dense convolutional neural networks

Sentinel-2 satellites have become one of the main resources for Earth observation images because they are free of charge, have a great spatial coverage and high temporal revisit. Sentinel-2 senses the same location providing different spatial resolutions as well as generating a multi-spectral image with 13 bands of 10, 20, and 60 m/pixel. In this work, we propose a single-image super-resolution model based on convolutional neural networks that enhances the low-resolution bands (20 m and 60 m) to reach the maximal resolution sensed (10 m) at the same time, whereas other approaches provide two independent models for each group of LR bands. Our proposed model, named Sen2-RDSR, is made up of Residual in Residual blocks that produce two final outputs at maximal resolution, one for 20 m/pixel bands and the other for 60 m/pixel bands. The training is done in two stages, first focusing on 20 m bands and then on the 60 m bands. Experimental results using six quality metrics (RMSE, SRE, SAM, PSNR, SSIM, ERGAS) show that our model has superior performance compared to other state-of-the-art approaches, and it is very effective and suitable as a preliminary step for land and coastal applications, as studies involving pixel-based classification for Land-Use-Land-Cover or the generation of vegetation indices.This work was funded by the Spanish Agencia Estatal de Investigación (AEI) under projects ARTEMISAT-2 (CTM2016-77733-R) and PID2020-117142GB-I00 of the call MCIN/AEI/10.13039/501100011033).Peer ReviewedPostprint (published version

Color Image Enhancement Techniques for Endoscopic images

Modern endoscopes play an important role in diagnosing various gastrointestinal (GI) tract related diseases. Although clinical findings of modern endoscopic imaging techniques are encouraging, there still remains much room for improvement of image quality. Of greatest concern, endoscopic images suffer from various degradations, such as specular highlights, non-uniform brightness and poor contrast. As a result, gastroenterologists often face difficulty in successfully identifying the subtle features, such as mucosal surface and structures, pit patterns, size and pattern of micro-vessels, tissue and vascular characteristics, superficial layer of mucosal and abnormal growths in endoscopic images. The improved visual quality of images can provide better diagnosis. This paper presents two proposed post-processing techniques for enhancing the subtle features of endoscopic images. The first proposed technique is named as endoscopic image enhancement based on adaptive sigmoid function and space-variant color reproduction (ASSVCR). It is achieved in two stages: image enhancement at gray level followed by color reproduction with the help of space variant chrominance mapping. Image enhancement is achieved by performing adaptive sigmoid function and uniform distribution of sigmoid pixels. Then color reproduction is used to generate new chrominance components. The second proposed technique is named as tri-scan. It is achieved in three stages: (1) Tissue and surface enhancement: a modified linear unsharp masking is used to sharpen the surface and edges of tissue and vascular characteristics, (2) Mucosa layer enhancement: an adaptive sigmoid function similar to the ASSVCR technique is employed on the R plane of the image to highlight the superficial layers of mucosa, (3) Color tone enhancement: the pixels are uniformly distributed to create a different color effect to highlight mucosa structures, superficial layers of mucosa and tissue characteristics. Both techniques are compared with other related works. Several performance metrics like focus value, statistic of visual representation, measurement of uniform distribution, color similarity test, color enhancement factor (CEF) and time complexity are used to assess the performance. The results showed improved performance compared to similar existing methods. In the post-processed images, we have observed that the ASSVCR can enhance and highlight pit patterns, tissue and vascular characteristics, mucosa structures and abnormal growths. It cannot highlight size and pattern of micro-vessels, and superficial layer of mucosa. In contrast, tri-scan can enhance and highlight all above mentioned features of endoscopic images

Pansharpening of High and Medium Resolution Satellite Images Using Bilateral Filtering

We provide and evaluate a fusion algorithm of remotely sensed images, i.e. the fusion of a panchromatic (PAN) image with a multi-spectral (MS) image using bilateral filtering, applied to images of three different sensors: SPOT 5, Landsat ETM+ and Quickbird. To assess the fusion process, we use six quality indexes, that confirm, along with visual analysis, good overall results for the three sensors