10 research outputs found

    Target-adaptive CNN-based pansharpening

    Full text link
    We recently proposed a convolutional neural network (CNN) for remote sensing image pansharpening obtaining a significant performance gain over the state of the art. In this paper, we explore a number of architectural and training variations to this baseline, achieving further performance gains with a lightweight network which trains very fast. Leveraging on this latter property, we propose a target-adaptive usage modality which ensures a very good performance also in the presence of a mismatch w.r.t. the training set, and even across different sensors. The proposed method, published online as an off-the-shelf software tool, allows users to perform fast and high-quality CNN-based pansharpening of their own target images on general-purpose hardware

    DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion

    Full text link
    Denosing diffusion model, as a generative model, has received a lot of attention in the field of image generation recently, thanks to its powerful generation capability. However, diffusion models have not yet received sufficient research in the field of image fusion. In this article, we introduce diffusion model to the image fusion field, treating the image fusion task as image-to-image translation and designing two different conditional injection modulation modules (i.e., style transfer modulation and wavelet modulation) to inject coarse-grained style information and fine-grained high-frequency and low-frequency information into the diffusion UNet, thereby generating fused images. In addition, we also discussed the residual learning and the selection of training objectives of the diffusion model in the image fusion task. Extensive experimental results based on quantitative and qualitative assessments compared with benchmarks demonstrates state-of-the-art results and good generalization performance in image fusion tasks. Finally, it is hoped that our method can inspire other works and gain insight into this field to better apply the diffusion model to image fusion tasks. Code shall be released for better reproducibility

    A review of spatial enhancement of hyperspectral remote sensing imaging techniques

    Get PDF
    Remote sensing technology has undeniable importance in various industrial applications, such as mineral exploration, plant detection, defect detection in aerospace and shipbuilding, and optical gas imaging, to name a few. Remote sensing technology has been continuously evolving, offering a range of image modalities that can facilitate the aforementioned applications. One such modality is Hyperspectral Imaging (HSI). Unlike Multispectral Images (MSI) and natural images, HSI consist of hundreds of bands. Despite their high spectral resolution, HSI suffer from low spatial resolution in comparison to their MSI counterpart, which hinders the utilization of their full potential. Therefore, spatial enhancement, or Super Resolution (SR), of HSI is a classical problem that has been gaining rapid attention over the past two decades. The literature is rich with various SR algorithms that enhance the spatial resolution of HSI while preserving their spectral fidelity. This paper reviews and discusses the most important algorithms relevant to this area of research between 2002-2022, along with the most frequently used datasets, HSI sensors, and quality metrics. Meta-analysis are drawn based on the aforementioned information, which is used as a foundation that summarizes the state of the field in a way that bridges the past and the present, identifies the current gap in it, and recommends possible future directions

    Single image super resolution for spatial enhancement of hyperspectral remote sensing imagery

    Get PDF
    Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations.Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations

    Super Resolution of Wavelet-Encoded Images and Videos

    Get PDF
    In this dissertation, we address the multiframe super resolution reconstruction problem for wavelet-encoded images and videos. The goal of multiframe super resolution is to obtain one or more high resolution images by fusing a sequence of degraded or aliased low resolution images of the same scene. Since the low resolution images may be unaligned, a registration step is required before super resolution reconstruction. Therefore, we first explore in-band (i.e. in the wavelet-domain) image registration; then, investigate super resolution. Our motivation for analyzing the image registration and super resolution problems in the wavelet domain is the growing trend in wavelet-encoded imaging, and wavelet-encoding for image/video compression. Due to drawbacks of widely used discrete cosine transform in image and video compression, a considerable amount of literature is devoted to wavelet-based methods. However, since wavelets are shift-variant, existing methods cannot utilize wavelet subbands efficiently. In order to overcome this drawback, we establish and explore the direct relationship between the subbands under a translational shift, for image registration and super resolution. We then employ our devised in-band methodology, in a motion compensated video compression framework, to demonstrate the effective usage of wavelet subbands. Super resolution can also be used as a post-processing step in video compression in order to decrease the size of the video files to be compressed, with downsampling added as a pre-processing step. Therefore, we present a video compression scheme that utilizes super resolution to reconstruct the high frequency information lost during downsampling. In addition, super resolution is a crucial post-processing step for satellite imagery, due to the fact that it is hard to update imaging devices after a satellite is launched. Thus, we also demonstrate the usage of our devised methods in enhancing resolution of pansharpened multispectral images

    Image Fusion in Remote Sensing and Quality Evaluation of Fused Images

    Get PDF
    In remote sensing, acquired optical images of high spectral resolution have usually a lower spatial resolution than images of lower spectral resolution. This is due to physical, cost and complexity constraints. To make the most of the available imagery, many image fusion techniques have been developed to address this problem. Image fusion is an ill-posed inverse problem where an image of low spatial resolution and high spectral resolution is enhanced in spatial-resolution by using an auxiliary image of high spatial resolution and low spectral resolution. It is assumed that both images display the same scene and are properly co-registered. Thus, the problem is essentially to transfer details from the higher spatial resolution auxiliary image to the upscaled lower resolution image in a manner that minimizes the spatial and spectral distortion of the fused image. The most common image fusion problem is pansharpening, where a multispectral (MS) image is enhanced using wide-band panchromatic (PAN) image. A similar problem is the enhancement of a hyperspectral (HS) image by either a PAN image or an MS image. As there is no reference image available, the reliable quantitative evaluation of the quality of the fused image is a difficult problem. This thesis addresses the image fusion problem in three different ways and also addresses the problem of quantitative quality evaluation.Í fjarkönnun hafa myndir með háa rófsupplausn lægri rúmupplausn en myndir með lægri rófsupplausn vegna eðlisfræðilegra og kostnaðarlegra takmarkana. Til að auka upplýsingamagn slíkra mynda hafa verið þróaðar fjölmargar sambræðsluaðferðir á síðustu tveimur áratugum. Myndsambræðsla er illa framsett andhverft vandmál (e. inverse problem) þar sem rúmupplausn myndar af hárri rófsupplausn er aukin með því að nota upplýsingar frá mynd af hárri rúmupplausn og lægri rófsupplausn. Það er gert ráð fyrir að báðar myndir sýni nákvæmlega sama landsvæði. Þannig er vandamálið í eðli sínu að flytja fíngerða eiginleika myndar af hærri rúmupplausn yfir á mynd af lægri rúmupplausn sem hefur verið brúuð upp í stærð hinnar myndarinnar, án þess að skerða gæði rófsupplýsinga upphaflegu myndarinnar. Algengasta myndbræðsluvandamálið í fjarkönnun er svokölluð panskerpun (e. pansharpening) þar sem fjölrásamynd (e. multispectral image) er endurbætt í rúmi með svokallaðri víðbandsmynd (e. panchromatic image) sem hefur aðeins eina rás af hárri upplausn. Annað svipað vandamál er sambræðsla háfjölrásamyndar (e. hyperspectral image) og annaðhvort fjölrásamyndar eða víðbandsmyndar. Þar sem myndsambræðsla er andhverft vandmál er engin háupplausnar samanburðarmynd tiltæk, sem gerir mat á gæðum sambræddu myndarinnar að erfiðu vandamáli. Í þessari ritgerð eru kynntar þrjár aðferðir sem taka á myndsambræðlsu og einnig er fjallað um mat á gæðum sambræddra mynda, þá sérstaklega panskerptra mynda

    Advances in Image Processing, Analysis and Recognition Technology

    Get PDF
    For many decades, researchers have been trying to make computers’ analysis of images as effective as the system of human vision is. For this purpose, many algorithms and systems have previously been created. The whole process covers various stages, including image processing, representation and recognition. The results of this work can be applied to many computer-assisted areas of everyday life. They improve particular activities and provide handy tools, which are sometimes only for entertainment, but quite often, they significantly increase our safety. In fact, the practical implementation of image processing algorithms is particularly wide. Moreover, the rapid growth of computational complexity and computer efficiency has allowed for the development of more sophisticated and effective algorithms and tools. Although significant progress has been made so far, many issues still remain, resulting in the need for the development of novel approaches
    corecore