Hyperspectral Image Restoration via Multi-mode and Double-weighted Tensor Nuclear Norm Minimization

Tensor nuclear norm (TNN) induced by tensor singular value decomposition plays an important role in hyperspectral image (HSI) restoration tasks. In this letter, we first consider three inconspicuous but crucial phenomenons in TNN. In the Fourier transform domain of HSIs, different frequency components contain different information; different singular values of each frequency component also represent different information. The two physical phenomenons lie not only in the spectral dimension but also in the spatial dimensions. Then, to improve the capability and flexibility of TNN for HSI restoration, we propose a multi-mode and double-weighted TNN based on the above three crucial phenomenons. It can adaptively shrink the frequency components and singular values according to their physical meanings in all modes of HSIs. In the framework of the alternating direction method of multipliers, we design an effective alternating iterative strategy to optimize our proposed model. Restoration experiments on both synthetic and real HSI datasets demonstrate their superiority against related methods

A General Destriping Framework for Remote Sensing Images Using Flatness Constraint

This paper proposes a general destriping framework using flatness constraints, where we can handle various regularization functions in a unified manner. Removing stripe noise, i.e., destriping, from remote sensing images is an essential task in terms of visual quality and subsequent processing. Most of the existing methods are designed by combining a particular image regularization with a stripe noise characterization that cooperates with the regularization, which precludes us to examine different regularizations to adapt to various target images. To resolve this, we formulate the destriping problem as a convex optimization problem involving a general form of image regularization and the flatness constraints, a newly introduced stripe noise characterization. This strong characterization enables us to consistently capture the nature of stripe noise, regardless of the choice of image regularization. For solving the optimization problem, we also develop an efficient algorithm based on a diagonally preconditioned primal-dual splitting algorithm (DP-PDS), which can automatically adjust the stepsizes. The effectiveness of our framework is demonstrated through destriping experiments, where we comprehensively compare combinations of image regularizations and stripe noise characterizations using hyperspectral images (HSI) and infrared (IR) videos.Comment: submitted to IEEE Transactions on Geoscience and Remote Sensin

Deep Plug-and-Play Prior for Hyperspectral Image Restoration

Deep-learning-based hyperspectral image (HSI) restoration methods have gained great popularity for their remarkable performance but often demand expensive network retraining whenever the specifics of task changes. In this paper, we propose to restore HSIs in a unified approach with an effective plug-and-play method, which can jointly retain the flexibility of optimization-based methods and utilize the powerful representation capability of deep neural networks. Specifically, we first develop a new deep HSI denoiser leveraging gated recurrent convolution units, short- and long-term skip connections, and an augmented noise level map to better exploit the abundant spatio-spectral information within HSIs. It, therefore, leads to the state-of-the-art performance on HSI denoising under both Gaussian and complex noise settings. Then, the proposed denoiser is inserted into the plug-and-play framework as a powerful implicit HSI prior to tackle various HSI restoration tasks. Through extensive experiments on HSI super-resolution, compressed sensing, and inpainting, we demonstrate that our approach often achieves superior performance, which is competitive with or even better than the state-of-the-art on each task, via a single model without any task-specific training.Comment: code at https://github.com/Zeqiang-Lai/DPHSI

Variable-Wise Diagonal Preconditioning for Primal-Dual Splitting: Design and Applications

This paper proposes a method of designing appropriate diagonal preconditioners for a preconditioned primal-dual splitting method (P-PDS). P-PDS can efficiently solve various types of convex optimization problems arising in signal processing and image processing. Since the appropriate diagonal preconditioners that accelerate the convergence of P-PDS vary greatly depending on the structure of the target optimization problem, a design method of diagonal preconditioners for PPDS has been proposed to determine them automatically from the problem structure. However, the existing method has two limitations: it requires direct access to all elements of the matrices representing the linear operators involved in the target optimization problem, and it is element-wise preconditioning, which makes certain types of proximity operators impossible to compute analytically. To overcome these limitations, we establish an Operator-norm-based design method of Variable-wise Diagonal Preconditioning (OVDP). First, the diagonal preconditioners constructed by OVDP are defined using only the operator norm or its upper bound of the linear operator thus eliminating the need for their explicit matrix representations. Furthermore, since our method is variable-wise preconditioning, it keeps all proximity operators efficiently computable. We also prove that our preconditioners satisfy the convergence conditions of PPDS. Finally, we demonstrate the effectiveness and utility of our method through applications to hyperspectral image mixed noise removal, hyperspectral unmixing, and graph signal recovery.Comment: Submitted to IEEE Transactions on Signal Processin

H2TF for Hyperspectral Image Denoising: Where Hierarchical Nonlinear Transform Meets Hierarchical Matrix Factorization

Recently, tensor singular value decomposition (t-SVD) has emerged as a promising tool for hyperspectral image (HSI) processing. In the t-SVD, there are two key building blocks: (i) the low-rank enhanced transform and (ii) the accompanying low-rank characterization of transformed frontal slices. Previous t-SVD methods mainly focus on the developments of (i), while neglecting the other important aspect, i.e., the exact characterization of transformed frontal slices. In this letter, we exploit the potentiality in both building blocks by leveraging the \underline{\bf H}ierarchical nonlinear transform and the \underline{\bf H}ierarchical matrix factorization to establish a new \underline{\bf T}ensor \underline{\bf F}actorization (termed as H2TF). Compared to shallow counter partners, e.g., low-rank matrix factorization or its convex surrogates, H2TF can better capture complex structures of transformed frontal slices due to its hierarchical modeling abilities. We then suggest the H2TF-based HSI denoising model and develop an alternating direction method of multipliers-based algorithm to address the resultant model. Extensive experiments validate the superiority of our method over state-of-the-art HSI denoising methods

Low-Rank Tensor Recovery with Euclidean-Norm-Induced Schatten-p Quasi-Norm Regularization

The nuclear norm and Schatten-$p$ quasi-norm of a matrix are popular rank proxies in low-rank matrix recovery. Unfortunately, computing the nuclear norm or Schatten-$p$ quasi-norm of a tensor is NP-hard, which is a pity for low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA). In this paper, we propose a new class of rank regularizers based on the Euclidean norms of the CP component vectors of a tensor and show that these regularizers are monotonic transformations of tensor Schatten-$p$ quasi-norm. This connection enables us to minimize the Schatten-$p$ quasi-norm in LRTC and TRPCA implicitly. The methods do not use the singular value decomposition and hence scale to big tensors. Moreover, the methods are not sensitive to the choice of initial rank and provide an arbitrarily sharper rank proxy for low-rank tensor recovery compared to nuclear norm. We provide theoretical guarantees in terms of recovery error for LRTC and TRPCA, which show relatively smaller $p$ of Schatten-$p$ quasi-norm leads to tighter error bounds. Experiments using LRTC and TRPCA on synthetic data and natural images verify the effectiveness and superiority of our methods compared to baseline methods

Sensor-independent LAI/FPAR CDR: reconstructing a global sensor-independent climate data record of MODIS and VIIRS LAI/FPAR from 2000 to 2022

Leaf area index (LAI) and fraction of photosynthetically active radiation (FPAR) are critical biophysical parameters for the characterization of terrestrial ecosystems. Long-term global LAI/FPAR products, such as the moderate resolution imaging spectroradiometer (MODIS) and the Visible Infrared Imaging Radiometer Suite (VIIRS), provide the fundamental dataset for accessing vegetation dynamics and studying climate change. However, existing global LAI/FPAR products suffer from several limitations, including spatial–temporal inconsistencies and accuracy issues. Considering these limitations, this study develops a sensor-independent (SI) LAI/FPAR climate data record (CDR) based on Terra-MODIS/Aqua-MODIS/VIIRS LAI/FPAR standard products. The SI LAI/FPAR CDR covers the period from 2000 to 2022, at spatial resolutions of 500 m/5 km/0.05∘, 8 d/bimonthly temporal frequencies and available in sinusoidal and WGS1984 projections. The methodology includes (i) comprehensive analyses of sensor-specific quality assessment variables to select high-quality retrievals, (ii) application of the spatial–temporal tensor (ST-tensor) completion model to extrapolate LAI and FPAR beyond areas with high-quality retrievals, (iii) generation of SI LAI/FPAR CDR in various projections and various spatial and temporal resolutions, and (iv) evaluation of the CDR by direct comparisons with ground data and indirectly through reproducing results of LAI/FPAR trends documented in the literature. This paper provides a comprehensive analysis of each step involved in the generation of the SI LAI/FPAR CDR, as well as evaluation of the ST-tensor completion model. Comparisons of SI LAI (FPAR) CDR with ground truth data suggest an RMSE of 0.84 LAI (0.15 FPAR) units with R2 of 0.72 (0.79), which outperform the standard Terra/Aqua/VIIRS LAI (FPAR) products. The SI LAI/FPAR CDR is characterized by a low time series stability (TSS) value, suggesting a more stable and less noisy dataset than sensor-dependent counterparts. Furthermore, the mean absolute error (MAE) of the CDR is also lower, suggesting that SI LAI/FPAR CDR is comparable in accuracy to high-quality retrievals. LAI/FPAR trend analyses based on the SI LAI/FPAR CDR agree with previous studies, which indirectly provides enhanced capabilities to utilize this CDR for studying vegetation dynamics and climate change. Overall, the integration of multiple satellite data sources and the use of advanced gap filling modeling techniques improve the accuracy of the SI LAI/FPAR CDR, ensuring the reliability of long-term vegetation studies, global carbon cycle modeling, and land policy development for informed decision-making and sustainable environmental management. The SI LAI/FPAR CDR is open access and available under a Creative Commons Attribution 4.0 License at https://doi.org/10.5281/zenodo.8076540 (Pu et al., 2023a).</p