A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Realistic hyperspectral image (HSI) super-resolution (SR) techniques aim to generate a high-resolution (HR) HSI with higher spectral and spatial fidelity from its low-resolution (LR) counterpart. The generative adversarial network (GAN) has proven to be an effective deep learning framework for image super-resolution. However, the optimisation process of existing GAN-based models frequently suffers from the problem of mode collapse, leading to the limited capacity of spectral-spatial invariant reconstruction. This may cause the spectral-spatial distortion on the generated HSI, especially with a large upscaling factor. To alleviate the problem of mode collapse, this work has proposed a novel GAN model coupled with a latent encoder (LE-GAN), which can map the generated spectral-spatial features from the image space to the latent space and produce a coupling component to regularise the generated samples. Essentially, we treat an HSI as a high-dimensional manifold embedded in a latent space. Thus, the optimisation of GAN models is converted to the problem of learning the distributions of high-resolution HSI samples in the latent space, making the distributions of the generated super-resolution HSIs closer to those of their original high-resolution counterparts. We have conducted experimental evaluations on the model performance of super-resolution and its capability in alleviating mode collapse. The proposed approach has been tested and validated based on two real HSI datasets with different sensors (i.e. AVIRIS and UHD-185) for various upscaling factors and added noise levels, and compared with the state-of-the-art super-resolution models (i.e. HyCoNet, LTTR, BAGAN, SR- GAN, WGAN).Comment: 18 pages, 10 figure

Spectral Super-Resolution of Satellite Imagery with Generative Adversarial Networks

Hyperspectral (HS) data is the most accurate interpretation of surface as it provides fine spectral information with hundreds of narrow contiguous bands as compared to multispectral (MS) data whose bands cover bigger wavelength portions of the electromagnetic spectrum. This difference is noticeable in applications such as agriculture, geosciences, astronomy, etc. However, HS sensors lack on earth observing spacecraft due to its high cost. In this study, we propose a novel loss function for generative adversarial networks as a spectral-oriented and general-purpose solution to spectral super-resolution of satellite imagery. The proposed architecture learns mapping from MS to HS data, generating nearly 20x more bands than the given input. We show that we outperform the state-of-the-art methods by visual interpretation and statistical metrics.Les dades hiperspectrals (HS) són la interpretació més precisa de la superfície, ja que proporciona informació espectral fina amb centenars de bandes contigües estretes en comparació amb les dades multiespectrals (MS) les bandes cobreixen parts de longitud d'ona més grans de l'espectre electromagnètic. Aquesta diferència és notable en àmbits com l'agricultura, les geociències, l'astronomia, etc. No obstant això, els sensors HS manquen als satèl·lits d'observació terrestre a causa del seu elevat cost. En aquest estudi proposem una nova funció de cost per a Generative Adversarial Networks com a solució orientada a l'espectre i de propòsit general per la superresolució espectral d'imatges de satèl·lit. L'arquitectura proposada aprèn el mapatge de dades MS a HS, generant gairebé 20x més bandes que l'entrada donada. Mostrem que superem els mètodes state-of-the-art mitjançant la interpretació visual i les mètriques estadístiques.Los datos hiperspectral (HS) son la interpretación más precisa de la superficie, ya que proporciona información espectral fina con cientos de bandas contiguas estrechas en comparación con los datos multiespectrales (MS) cuyas bandas cubren partes de longitud de onda más grandes del espectro electromagnético. Esta diferencia es notable en ámbitos como la agricultura, las geociencias, la astronomía, etc. Sin embargo, los sensores HS escasean en los satélites de observación terrestre debido a su elevado coste. En este estudio proponemos una nueva función de coste para Generative Adversarial Networks como solución orientada al espectro y de propósito general para la super-resolución espectral de imágenes de satélite. La arquitectura propuesta aprende el mapeo de datos MS a HS, generando casi 20x más bandas que la entrada dada. Mostramos que superamos los métodos state-of-the-art mediante la interpretación visual y las métricas estadísticas

Learning spatial and spectral features via 2D-1D generative adversarial network for hyperspectral image super-resolution

Three-dimensional (3D) convolutional networks have been proven to be able to explore spatial context and spectral information simultaneously for super-resolution (SR). However, such kind of network can’t be practically designed very ‘deep’ due to the long training time and GPU memory limitations involved in 3D convolution. Instead, in this paper, spatial context and spectral information in hyperspectral images (HSIs) are explored using Two-dimensional (2D) and Onedimenional (1D) convolution, separately. Therefore, a novel 2D-1D generative adversarial network architecture (2D-1DHSRGAN) is proposed for SR of HSIs. Specifically, the generator network consists of a spatial network and a spectral network, in which spatial network is trained with the least absolute deviations loss function to explore spatial context by 2D convolution and spectral network is trained with the spectral angle mapper (SAM) loss function to extract spectral information by 1D convolution. Experimental results over two real HSIs demonstrate that the proposed 2D-1D-HSRGAN clearly outperforms several state-of-the-art algorithms

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

DALF: An AI Enabled Adversarial Framework for Classification of Hyperspectral Images

Hyperspectral image classification is very complex and challenging process. However, with deep neural networks like Convolutional Neural Networks (CNN) with explicit dimensionality reduction, the capability of classifier is greatly increased. However, there is still problem with sufficient training samples. In this paper, we overcome this problem by proposing an Artificial Intelligence (AI) based framework named Deep Adversarial Learning Framework (DALF) that exploits deep autoencoder for dimensionality reduction, Generative Adversarial Network (GAN) for generating new Hyperspectral Imaging (HSI) samples that are to be verified by a discriminator in a non-cooperative game setting besides using aclassifier. Convolutional Neural Network (CNN) is used for both generator and discriminator while classifier role is played by Support Vector Machine (SVM) and Neural Network (NN). An algorithm named Generative Model based Hybrid Approach for HSI Classification (GMHA-HSIC) which drives the functionality of the proposed framework is proposed. The success of DALF in accurate classification is largely dependent on the synthesis and labelling of spectra on regular basis. The synthetic samples made with an iterative process and being verified by discriminator result in useful spectra. By training GAN with associated deep learning models, the framework leverages classification performance. Our experimental results revealed that the proposed framework has potential to improve the state of the art besides having an effective data augmentation strategy

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

Modern hyperspectral imaging systems produce huge datasets potentially conveying a great abundance of information; such a resource, however, poses many challenges in the analysis and interpretation of these data. Deep learning approaches certainly offer a great variety of opportunities for solving classical imaging tasks and also for approaching new stimulating problems in the spatial–spectral domain. This is fundamental in the driving sector of Remote Sensing where hyperspectral technology was born and has mostly developed, but it is perhaps even more true in the multitude of current and evolving application sectors that involve these imaging technologies. The present review develops on two fronts: on the one hand, it is aimed at domain professionals who want to have an updated overview on how hyperspectral acquisition techniques can combine with deep learning architectures to solve specific tasks in different application fields. On the other hand, we want to target the machine learning and computer vision experts by giving them a picture of how deep learning technologies are applied to hyperspectral data from a multidisciplinary perspective. The presence of these two viewpoints and the inclusion of application fields other than Remote Sensing are the original contributions of this review, which also highlights some potentialities and critical issues related to the observed development trends

Learnable Reconstruction Methods from RGB Images to Hyperspectral Imaging: A Survey

Hyperspectral imaging enables versatile applications due to its competence in capturing abundant spatial and spectral information, which are crucial for identifying substances. However, the devices for acquiring hyperspectral images are expensive and complicated. Therefore, many alternative spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from lower-cost, more available RGB images. We present a thorough investigation of these state-of-the-art spectral reconstruction methods from the widespread RGB images. A systematic study and comparison of more than 25 methods has revealed that most of the data-driven deep learning methods are superior to prior-based methods in terms of reconstruction accuracy and quality despite lower speeds. This comprehensive review can serve as a fruitful reference source for peer researchers, thus further inspiring future development directions in related domains