49 research outputs found
DiffSCI: Zero-Shot Snapshot Compressive Imaging via Iterative Spectral Diffusion Model
This paper endeavors to advance the precision of snapshot compressive imaging
(SCI) reconstruction for multispectral image (MSI). To achieve this, we
integrate the advantageous attributes of established SCI techniques and an
image generative model, propose a novel structured zero-shot diffusion model,
dubbed DiffSCI. DiffSCI leverages the structural insights from the deep prior
and optimization-based methodologies, complemented by the generative
capabilities offered by the contemporary denoising diffusion model.
Specifically, firstly, we employ a pre-trained diffusion model, which has been
trained on a substantial corpus of RGB images, as the generative denoiser
within the Plug-and-Play framework for the first time. This integration allows
for the successful completion of SCI reconstruction, especially in the case
that current methods struggle to address effectively. Secondly, we
systematically account for spectral band correlations and introduce a robust
methodology to mitigate wavelength mismatch, thus enabling seamless adaptation
of the RGB diffusion model to MSIs. Thirdly, an accelerated algorithm is
implemented to expedite the resolution of the data subproblem. This
augmentation not only accelerates the convergence rate but also elevates the
quality of the reconstruction process. We present extensive testing to show
that DiffSCI exhibits discernible performance enhancements over prevailing
self-supervised and zero-shot approaches, surpassing even supervised
transformer counterparts across both simulated and real datasets. Our code will
be available
Inheriting Bayer's Legacy-Joint Remosaicing and Denoising for Quad Bayer Image Sensor
Pixel binning based Quad sensors have emerged as a promising solution to
overcome the hardware limitations of compact cameras in low-light imaging.
However, binning results in lower spatial resolution and non-Bayer CFA
artifacts. To address these challenges, we propose a dual-head joint
remosaicing and denoising network (DJRD), which enables the conversion of noisy
Quad Bayer and standard noise-free Bayer pattern without any resolution loss.
DJRD includes a newly designed Quad Bayer remosaicing (QB-Re) block, integrated
denoising modules based on Swin-transformer and multi-scale wavelet transform.
The QB-Re block constructs the convolution kernel based on the CFA pattern to
achieve a periodic color distribution in the perceptual field, which is used to
extract exact spectral information and reduce color misalignment. The
integrated Swin-Transformer and multi-scale wavelet transform capture non-local
dependencies, frequency and location information to effectively reduce
practical noise. By identifying challenging patches utilizing Moire and zipper
detection metrics, we enable our model to concentrate on difficult patches
during the post-training phase, which enhances the model's performance in hard
cases. Our proposed model outperforms competing models by approximately 3dB,
without additional complexity in hardware or software
From model-based optimization algorithms to deep learning models for clustering hyperspectral images
Hyperspectral images (HSIs), captured by different Earth observation airborne and space-borne systems, provide rich spectral information in hundreds of bands, enabling far better discrimination between ground materials that are often indistinguishable in visible and multi-spectral images. Clustering of HSIs, which aims to unveil class patterns in an unsupervised way, is highly important in the interpretation of HSI, especially when labelled data are not available. A number of HSI clustering methods have been proposed. Among them, model-based optimization algorithms, which learn the cluster structure of data by solving convex/non-convex optimization problems, have achieved the current state-of-the-art performance. Recent works extend the model-based algorithms to deep versions with deep neural networks, obtaining huge breakthroughs in clustering performance. However, a systematic survey on the topic is absent. This article provides a comprehensive overview of clustering methods of HSI and tracked the latest techniques and breakthroughs in the domain, including the traditional model-based optimization algorithms and the emerging deep learning based clustering methods. With a new taxonomy, we elaborated on the main ideas, technical details, advantages, and disadvantages of different types of clustering methods of HSIs. We provided a systematic performance comparison between different clustering methods by conducting extensive experiments on real HSIs. Unsolved problems and future research trends in the domain are pointed out. Moreover, we provided a toolbox that contains implementations of representative clustering algorithms to help researchers to develop their own models