1,334 research outputs found
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
Plug-and-play (PnP) is a non-convex framework that integrates modern
denoising priors, such as BM3D or deep learning-based denoisers, into ADMM or
other proximal algorithms. An advantage of PnP is that one can use pre-trained
denoisers when there is not sufficient data for end-to-end training. Although
PnP has been recently studied extensively with great empirical success,
theoretical analysis addressing even the most basic question of convergence has
been insufficient. In this paper, we theoretically establish convergence of
PnP-FBS and PnP-ADMM, without using diminishing stepsizes, under a certain
Lipschitz condition on the denoisers. We then propose real spectral
normalization, a technique for training deep learning-based denoisers to
satisfy the proposed Lipschitz condition. Finally, we present experimental
results validating the theory.Comment: Published in the International Conference on Machine Learning, 201
Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
We consider the reconstruction problem of video snapshot compressive imaging
(SCI), which captures high-speed videos using a low-speed 2D sensor (detector).
The underlying principle of SCI is to modulate sequential high-speed frames
with different masks and then these encoded frames are integrated into a
snapshot on the sensor and thus the sensor can be of low-speed. On one hand,
video SCI enjoys the advantages of low-bandwidth, low-power and low-cost. On
the other hand, applying SCI to large-scale problems (HD or UHD videos) in our
daily life is still challenging and one of the bottlenecks lies in the
reconstruction algorithm. Exiting algorithms are either too slow (iterative
optimization algorithms) or not flexible to the encoding process (deep learning
based end-to-end networks). In this paper, we develop fast and flexible
algorithms for SCI based on the plug-and-play (PnP) framework. In addition to
the PnP-ADMM method, we further propose the PnP-GAP (generalized alternating
projection) algorithm with a lower computational workload. We first employ the
image deep denoising priors to show that PnP can recover a UHD color video with
30 frames from a snapshot measurement. Since videos have strong temporal
correlation, by employing the video deep denoising priors, we achieve a
significant improvement in the results. Furthermore, we extend the proposed PnP
algorithms to the color SCI system using mosaic sensors, where each pixel only
captures the red, green or blue channels. A joint reconstruction and
demosaicing paradigm is developed for flexible and high quality reconstruction
of color video SCI systems. Extensive results on both simulation and real
datasets verify the superiority of our proposed algorithm.Comment: 18 pages, 12 figures and 4 tables. Journal extension of
arXiv:2003.13654. Code available at
https://github.com/liuyang12/PnP-SCI_pytho
Structured Kernel Estimation for Photon-Limited Deconvolution
Images taken in a low light condition with the presence of camera shake
suffer from motion blur and photon shot noise. While state-of-the-art image
restoration networks show promising results, they are largely limited to
well-illuminated scenes and their performance drops significantly when photon
shot noise is strong.
In this paper, we propose a new blur estimation technique customized for
photon-limited conditions. The proposed method employs a gradient-based
backpropagation method to estimate the blur kernel. By modeling the blur kernel
using a low-dimensional representation with the key points on the motion
trajectory, we significantly reduce the search space and improve the regularity
of the kernel estimation problem. When plugged into an iterative framework, our
novel low-dimensional representation provides improved kernel estimates and
hence significantly better deconvolution performance when compared to
end-to-end trained neural networks. The source code and pretrained models are
available at \url{https://github.com/sanghviyashiitb/structured-kernel-cvpr23}Comment: main document and supplementary; accepted at CVPR202
- …