On-line blind unmixing for hyperspectral pushbroom imaging systems

International audienceIn this paper, the on-line hyperspectral image blind unmixing is addressed. Inspired by the Incremental Non-negative Matrix Factorization (INMF) method, we propose an on-line NMF which is adapted to the acquisition scheme of a pushbroom imager. Because of the non-uniqueness of the NMF model, a minimum volume constraint on the endmembers is added allowing to reduce the set of admissible solutions. This results in a stable algorithm yielding results similar to those of standard off-line NMF methods, but drastically reducing the computation time. The algorithm is applied to wood hyperspectral images showing that such a technique is effective for the on-line prediction of wood piece rendering after finishing. Index Terms— Hyperspectral imaging, Pushbroom imager, On-line Non-negative Matrix Factorization, Minimum volume constraint

Early identification of mushy Halibut syndrome with hyperspectral image analysis

Mushy Halibut Syndrome (MHS) is a condition that appears in Greenland halibut and manifests itself as abnormally opaque, flaccid and jelly-like flesh. Fish affected by this syndrome show poor meat quality, which results in negative consequences for the fish industry. The research community has not carefully investigated this condition, nor novel technologies for MHS detection have been proposed. In this research work, we propose using hyperspectral imaging to detect MHS. After collecting a dataset of hyperspectral images of halibut affected by MHS, two different goals were targeted. Firstly, the estimation of the chemical composition of the samples (specifically fat and water content) from their spectral data by using constrained spectral unmixing. Secondly, supervised classification using partial least squares discriminant analysis (PLS-DA) was evaluated to identify specimens affected by MHS. The outcomes of our study suggest that the prediction of fat from the spectral data is possible, but the prediction of the water content was not found to be accurate. However, the detection of MHS using PLS-DA was precise for hyperspectral images from both fillets and whole fish, with lower bounds of 75% and 83% for precision and recall, respectively. Our findings suggest hyperspectral imaging as a suitable technology for the early screening of MHS.Early identification of mushy Halibut syndrome with hyperspectral image analysispublishedVersio

Early identification of mushy Halibut syndrome with hyperspectral image analysis

Mushy Halibut Syndrome (MHS) is a condition that appears in Greenland halibut and manifests itself as abnormally opaque, flaccid and jelly-like flesh. Fish affected by this syndrome show poor meat quality, which results in negative consequences for the fish industry. The research community has not carefully investigated this condition, nor novel technologies for MHS detection have been proposed. In this research work, we propose using hyperspectral imaging to detect MHS. After collecting a dataset of hyperspectral images of halibut affected by MHS, two different goals were targeted. Firstly, the estimation of the chemical composition of the samples (specifically fat and water content) from their spectral data by using constrained spectral unmixing. Secondly, supervised classification using partial least squares discriminant analysis (PLS-DA) was evaluated to identify specimens affected by MHS. The outcomes of our study suggest that the prediction of fat from the spectral data is possible, but the prediction of the water content was not found to be accurate. However, the detection of MHS using PLS-DA was precise for hyperspectral images from both fillets and whole fish, with lower bounds of 75% and 83% for precision and recall, respectively. Our findings suggest hyperspectral imaging as a suitable technology for the early screening of MHS.Early identification of mushy Halibut syndrome with hyperspectral image analysispublishedVersio

Hyperspectral Super-Resolution with Coupled Tucker Approximation: Recoverability and SVD-based algorithms

We propose a novel approach for hyperspectral super-resolution, that is based on low-rank tensor approximation for a coupled low-rank multilinear (Tucker) model. We show that the correct recovery holds for a wide range of multilinear ranks. For coupled tensor approximation, we propose two SVD-based algorithms that are simple and fast, but with a performance comparable to the state-of-the-art methods. The approach is applicable to the case of unknown spatial degradation and to the pansharpening problem.Comment: IEEE Transactions on Signal Processing, Institute of Electrical and Electronics Engineers, in Pres

Single image super resolution for spatial enhancement of hyperspectral remote sensing imagery

Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations.Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations

Démélange d'images hyperspectrales à l'aide de la NMF en-ligne avec contrainte de dispersion minimale

National audienceWe propose a new method for on-line hyperspectral images unmixing based on an ADMM (Alternating Direction Method of Multipliers) approach, particularly well-adapted to pushbroom imaging systems. The proposed algorithm presents a faster convergence and a lower computational complexity compared to the algorithms based on multiplicative update rules. Due to the general ill-posed nature of the unmixing problem, we impose a minimal endmembers dispersion constraint; this constraint can be interpreted as a convex relaxation of the minimal volume constraint. Real data tests illustrate the good performance of the proposed method compared to the state of the art.Nous proposons une nouvelle méthode de démélange en-ligne d'images hyperspectrales fondée sur une approche de type ADMM (Alternating Direction Method of Multipliers), particulièrement bien adaptée aux systèmes d'imagerie pushbroom. L'algorithme proposé présente une convergence plus rapide et une complexité de calcul plus faible par rapport aux algorithmes fondés sur des règles de mise à jour multiplicatives. En raison du caractère généralement mal posé du problème de démélange, nous intégrons dans la méthode une contrainte de dispersion minimale des endmembers ; cette contrainte peut être interprétée comme une relaxation convexe de la contrainte de volume minimal. Des tests sur des données réelles permettent d'illustrer les bonnes performances de la méthode proposée comparées à l'état de l'art

Image Restoration for Remote Sensing: Overview and Toolbox

Remote sensing provides valuable information about objects or areas from a distance in either active (e.g., RADAR and LiDAR) or passive (e.g., multispectral and hyperspectral) modes. The quality of data acquired by remotely sensed imaging sensors (both active and passive) is often degraded by a variety of noise types and artifacts. Image restoration, which is a vibrant field of research in the remote sensing community, is the task of recovering the true unknown image from the degraded observed image. Each imaging sensor induces unique noise types and artifacts into the observed image. This fact has led to the expansion of restoration techniques in different paths according to each sensor type. This review paper brings together the advances of image restoration techniques with particular focuses on synthetic aperture radar and hyperspectral images as the most active sub-fields of image restoration in the remote sensing community. We, therefore, provide a comprehensive, discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to investigate the vibrant topic of data restoration by supplying sufficient detail and references. Additionally, this review paper accompanies a toolbox to provide a platform to encourage interested students and researchers in the field to further explore the restoration techniques and fast-forward the community. The toolboxes are provided in https://github.com/ImageRestorationToolbox.Comment: This paper is under review in GRS

Hyperspectral Imaging for Landmine Detection

This PhD thesis aims at investigating the possibility to detect landmines using hyperspectral imaging. Using this technology, we are able to acquire at each pixel of the image spectral data in hundreds of wavelengths. So, at each pixel we obtain a reflectance spectrum that is used as fingerprint to identify the materials in each pixel, and mainly in our project help us to detect the presence of landmines. The proposed process works as follows: a preconfigured drone (hexarotor or octorotor) will carry the hyperspectral camera. This programmed drone is responsible of flying over the contaminated area in order to take images from a safe distance. Various image processing techniques will be used to treat the image in order to isolate the landmine from the surrounding. Once the presence of a mine or explosives is suspected, an alarm signal is sent to the base station giving information about the type of the mine, its location and the clear path that could be taken by the mine removal team in order to disarm the mine. This technology has advantages over the actually used techniques: • It is safer because it limits the need of humans in the searching process and gives the opportunity to the demining team to detect the mines while they are in a safe region. • It is faster. A larger area could be cleared in a single day by comparison with demining techniques • This technique can be used to detect at the same time objects other than mines such oil or minerals. First, a presentation of the problem of landmines that is expanding worldwide referring to some statistics from the UN organizations is provided. In addition, a brief presentation of different types of landmines is shown. Unfortunately, new landmines are well camouflaged and are mainly made of plastic in order to make their detection using metal detectors harder. A summary of all landmine detection techniques is shown to give an idea about the advantages and disadvantages of each technique. In this work, we give an overview of different projects that worked on the detection of landmines using hyperspectral imaging. We will show the main results achieved in this field and future work to be done in order to make this technology effective. Moreover, we worked on different target detection algorithms in order to achieve high probability of detection with low false alarm rate. We tested different statistical and linear unmixing based methods. In addition, we introduced the use of radial basis function neural networks in order to detect landmines at subpixel level. A comparative study between different detection methods will be shown in the thesis. A study of the effect of dimensionality reduction using principal component analysis prior to classification is also provided. The study shows the dependency between the two steps (feature extraction and target detection). The selection of target detection algorithm will define if feature extraction in previous phase is necessary. A field experiment has been done in order to study how the spectral signature of landmine will change depending on the environment in which the mine is planted. For this, we acquired the spectral signature of 6 types of landmines in different conditions: in Lab where specific source of light is used; in field where mines are covered by grass; and when mines are buried in soil. The results of this experiment are very interesting. The signature of two types of landmines are used in the simulations. They are a database necessary for supervised detection of landmines. Also we extracted some spectral characteristics of landmines that would help us to distinguish mines from background

Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials