175 research outputs found
Generalized Inpainting Method for Hyperspectral Image Acquisition
A recently designed hyperspectral imaging device enables multiplexed
acquisition of an entire data volume in a single snapshot thanks to
monolithically-integrated spectral filters. Such an agile imaging technique
comes at the cost of a reduced spatial resolution and the need for a
demosaicing procedure on its interleaved data. In this work, we address both
issues and propose an approach inspired by recent developments in compressed
sensing and analysis sparse models. We formulate our superresolution and
demosaicing task as a 3-D generalized inpainting problem. Interestingly, the
target spatial resolution can be adjusted for mitigating the compression level
of our sensing. The reconstruction procedure uses a fast greedy method called
Pseudo-inverse IHT. We also show on simulations that a random arrangement of
the spectral filters on the sensor is preferable to regular mosaic layout as it
improves the quality of the reconstruction. The efficiency of our technique is
demonstrated through numerical experiments on both synthetic and real data as
acquired by the snapshot imager.Comment: Keywords: Hyperspectral, inpainting, iterative hard thresholding,
sparse models, CMOS, Fabry-P\'ero
Low-Cost Compressive Sensing for Color Video and Depth
A simple and inexpensive (low-power and low-bandwidth) modification is made
to a conventional off-the-shelf color video camera, from which we recover
{multiple} color frames for each of the original measured frames, and each of
the recovered frames can be focused at a different depth. The recovery of
multiple frames for each measured frame is made possible via high-speed coding,
manifested via translation of a single coded aperture; the inexpensive
translation is constituted by mounting the binary code on a piezoelectric
device. To simultaneously recover depth information, a {liquid} lens is
modulated at high speed, via a variable voltage. Consequently, during the
aforementioned coding process, the liquid lens allows the camera to sweep the
focus through multiple depths. In addition to designing and implementing the
camera, fast recovery is achieved by an anytime algorithm exploiting the
group-sparsity of wavelet/DCT coefficients.Comment: 8 pages, CVPR 201
Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
We consider the reconstruction problem of video snapshot compressive imaging
(SCI), which captures high-speed videos using a low-speed 2D sensor (detector).
The underlying principle of SCI is to modulate sequential high-speed frames
with different masks and then these encoded frames are integrated into a
snapshot on the sensor and thus the sensor can be of low-speed. On one hand,
video SCI enjoys the advantages of low-bandwidth, low-power and low-cost. On
the other hand, applying SCI to large-scale problems (HD or UHD videos) in our
daily life is still challenging and one of the bottlenecks lies in the
reconstruction algorithm. Exiting algorithms are either too slow (iterative
optimization algorithms) or not flexible to the encoding process (deep learning
based end-to-end networks). In this paper, we develop fast and flexible
algorithms for SCI based on the plug-and-play (PnP) framework. In addition to
the PnP-ADMM method, we further propose the PnP-GAP (generalized alternating
projection) algorithm with a lower computational workload. We first employ the
image deep denoising priors to show that PnP can recover a UHD color video with
30 frames from a snapshot measurement. Since videos have strong temporal
correlation, by employing the video deep denoising priors, we achieve a
significant improvement in the results. Furthermore, we extend the proposed PnP
algorithms to the color SCI system using mosaic sensors, where each pixel only
captures the red, green or blue channels. A joint reconstruction and
demosaicing paradigm is developed for flexible and high quality reconstruction
of color video SCI systems. Extensive results on both simulation and real
datasets verify the superiority of our proposed algorithm.Comment: 18 pages, 12 figures and 4 tables. Journal extension of
arXiv:2003.13654. Code available at
https://github.com/liuyang12/PnP-SCI_pytho
InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments
Classic designs of hyperspectral instrumentation densely sample the spatial
and spectral information of the scene of interest. Data may be compressed after
the acquisition. In this paper we introduce a framework for the design of an
optimized, micro-patterned snapshot hyperspectral imager that acquires an
optimized subset of the spatial and spectral information in the scene. The data
is thereby compressed already at the sensor level, but can be restored to the
full hyperspectral data cube by the jointly optimized reconstructor. This
framework is implemented with TensorFlow and makes use of its automatic
differentiation for the joint optimization of the layout of the micro-patterned
filter array as well as the reconstructor. We explore the achievable
compression ratio for different numbers of filter passbands, number of scanning
frames, and filter layouts using data collected by the Hyperscout instrument.
We show resulting instrument designs that take snapshot measurements without
losing significant information while reducing the data volume, acquisition
time, or detector space by a factor of 40 as compared to classic, dense
sampling. The joint optimization of a compressive hyperspectral imager design
and the accompanying reconstructor provides an avenue to substantially reduce
the data volume from hyperspectral imagers.Comment: 23 pages, 12 figures, published in Applied Optic
Discriminative Transfer Learning for General Image Restoration
Recently, several discriminative learning approaches have been proposed for
effective image restoration, achieving convincing trade-off between image
quality and computational efficiency. However, these methods require separate
training for each restoration task (e.g., denoising, deblurring, demosaicing)
and problem condition (e.g., noise level of input images). This makes it
time-consuming and difficult to encompass all tasks and conditions during
training. In this paper, we propose a discriminative transfer learning method
that incorporates formal proximal optimization and discriminative learning for
general image restoration. The method requires a single-pass training and
allows for reuse across various problems and conditions while achieving an
efficiency comparable to previous discriminative approaches. Furthermore, after
being trained, our model can be easily transferred to new likelihood terms to
solve untrained tasks, or be combined with existing priors to further improve
image restoration quality
A Framework for Fast Image Deconvolution with Incomplete Observations
In image deconvolution problems, the diagonalization of the underlying
operators by means of the FFT usually yields very large speedups. When there
are incomplete observations (e.g., in the case of unknown boundaries), standard
deconvolution techniques normally involve non-diagonalizable operators,
resulting in rather slow methods, or, otherwise, use inexact convolution
models, resulting in the occurrence of artifacts in the enhanced images. In
this paper, we propose a new deconvolution framework for images with incomplete
observations that allows us to work with diagonalized convolution operators,
and therefore is very fast. We iteratively alternate the estimation of the
unknown pixels and of the deconvolved image, using, e.g., an FFT-based
deconvolution method. This framework is an efficient, high-quality alternative
to existing methods of dealing with the image boundaries, such as edge
tapering. It can be used with any fast deconvolution method. We give an example
in which a state-of-the-art method that assumes periodic boundary conditions is
extended, through the use of this framework, to unknown boundary conditions.
Furthermore, we propose a specific implementation of this framework, based on
the alternating direction method of multipliers (ADMM). We provide a proof of
convergence for the resulting algorithm, which can be seen as a "partial" ADMM,
in which not all variables are dualized. We report experimental comparisons
with other primal-dual methods, where the proposed one performed at the level
of the state of the art. Four different kinds of applications were tested in
the experiments: deconvolution, deconvolution with inpainting, superresolution,
and demosaicing, all with unknown boundaries.Comment: IEEE Trans. Image Process., to be published. 15 pages, 11 figures.
MATLAB code available at
https://github.com/alfaiate/DeconvolutionIncompleteOb
- …