190 research outputs found
Computational Spectral Imaging: A Contemporary Overview
Spectral imaging collects and processes information along spatial and
spectral coordinates quantified in discrete voxels, which can be treated as a
3D spectral data cube. The spectral images (SIs) allow identifying objects,
crops, and materials in the scene through their spectral behavior. Since most
spectral optical systems can only employ 1D or maximum 2D sensors, it is
challenging to directly acquire the 3D information from available commercial
sensors. As an alternative, computational spectral imaging (CSI) has emerged as
a sensing tool where the 3D data can be obtained using 2D encoded projections.
Then, a computational recovery process must be employed to retrieve the SI. CSI
enables the development of snapshot optical systems that reduce acquisition
time and provide low computational storage costs compared to conventional
scanning systems. Recent advances in deep learning (DL) have allowed the design
of data-driven CSI to improve the SI reconstruction or, even more, perform
high-level tasks such as classification, unmixing, or anomaly detection
directly from 2D encoded projections. This work summarises the advances in CSI,
starting with SI and its relevance; continuing with the most relevant
compressive spectral optical systems. Then, CSI with DL will be introduced, and
the recent advances in combining the physical optical design with computational
DL algorithms to solve high-level tasks
Quantum-inspired computational imaging
Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. These methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.Y.A. acknowledges support from the UK Royal Academy of Engineering under the Research Fellowship Scheme (RF201617/16/31). S.McL. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grant EP/J015180/1). V.G. acknowledges support from the U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office award W911NF-10-1-0404, the U.S. DARPA REVEAL program through contract HR0011-16-C-0030, and U.S. National Science Foundation through grants 1161413 and 1422034. A.H. acknowledges support from U.S. Army Research Office award W911NF-15-1-0479, U.S. Department of the Air Force grant FA8650-15-D-1845, and U.S. Department of Energy National Nuclear Security Administration grant DE-NA0002534. D.F. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grants EP/M006514/1 and EP/M01326X/1). (RF201617/16/31 - UK Royal Academy of Engineering; EP/J015180/1 - UK Engineering and Physical Sciences Research Council; EP/M006514/1 - UK Engineering and Physical Sciences Research Council; EP/M01326X/1 - UK Engineering and Physical Sciences Research Council; W911NF-10-1-0404 - U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office; HR0011-16-C-0030 - U.S. DARPA REVEAL program; 1161413 - U.S. National Science Foundation; 1422034 - U.S. National Science Foundation; W911NF-15-1-0479 - U.S. Army Research Office; FA8650-15-D-1845 - U.S. Department of the Air Force; DE-NA0002534 - U.S. Department of Energy National Nuclear Security Administration)Accepted manuscrip
A Non-Local Structure Tensor Based Approach for Multicomponent Image Recovery Problems
Non-Local Total Variation (NLTV) has emerged as a useful tool in variational
methods for image recovery problems. In this paper, we extend the NLTV-based
regularization to multicomponent images by taking advantage of the Structure
Tensor (ST) resulting from the gradient of a multicomponent image. The proposed
approach allows us to penalize the non-local variations, jointly for the
different components, through various matrix norms with .
To facilitate the choice of the hyper-parameters, we adopt a constrained convex
optimization approach in which we minimize the data fidelity term subject to a
constraint involving the ST-NLTV regularization. The resulting convex
optimization problem is solved with a novel epigraphical projection method.
This formulation can be efficiently implemented thanks to the flexibility
offered by recent primal-dual proximal algorithms. Experiments are carried out
for multispectral and hyperspectral images. The results demonstrate the
interest of introducing a non-local structure tensor regularization and show
that the proposed approach leads to significant improvements in terms of
convergence speed over current state-of-the-art methods
Compressive Source Separation: Theory and Methods for Hyperspectral Imaging
With the development of numbers of high resolution data acquisition systems
and the global requirement to lower the energy consumption, the development of
efficient sensing techniques becomes critical. Recently, Compressed Sampling
(CS) techniques, which exploit the sparsity of signals, have allowed to
reconstruct signal and images with less measurements than the traditional
Nyquist sensing approach. However, multichannel signals like Hyperspectral
images (HSI) have additional structures, like inter-channel correlations, that
are not taken into account in the classical CS scheme. In this paper we exploit
the linear mixture of sources model, that is the assumption that the
multichannel signal is composed of a linear combination of sources, each of
them having its own spectral signature, and propose new sampling schemes
exploiting this model to considerably decrease the number of measurements
needed for the acquisition and source separation. Moreover, we give theoretical
lower bounds on the number of measurements required to perform reconstruction
of both the multichannel signal and its sources. We also proposed optimization
algorithms and extensive experimentation on our target application which is
HSI, and show that our approach recovers HSI with far less measurements and
computational effort than traditional CS approaches.Comment: 32 page
Mixture-Net: Low-Rank Deep Image Prior Inspired by Mixture Models for Spectral Image Recovery
This paper proposes a non-data-driven deep neural network for spectral image
recovery problems such as denoising, single hyperspectral image
super-resolution, and compressive spectral imaging reconstruction. Unlike
previous methods, the proposed approach, dubbed Mixture-Net, implicitly learns
the prior information through the network. Mixture-Net consists of a deep
generative model whose layers are inspired by the linear and non-linear
low-rank mixture models, where the recovered image is composed of a weighted
sum between the linear and non-linear decomposition. Mixture-Net also provides
a low-rank decomposition interpreted as the spectral image abundances and
endmembers, helpful in achieving remote sensing tasks without running
additional routines. The experiments show the MixtureNet effectiveness
outperforming state-of-the-art methods in recovery quality with the advantage
of architecture interpretability
Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches
Imaging spectrometers measure electromagnetic energy scattered in their
instantaneous field view in hundreds or thousands of spectral channels with
higher spectral resolution than multispectral cameras. Imaging spectrometers
are therefore often referred to as hyperspectral cameras (HSCs). Higher
spectral resolution enables material identification via spectroscopic analysis,
which facilitates countless applications that require identifying materials in
scenarios unsuitable for classical spectroscopic analysis. Due to low spatial
resolution of HSCs, microscopic material mixing, and multiple scattering,
spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus,
accurate estimation requires unmixing. Pixels are assumed to be mixtures of a
few materials, called endmembers. Unmixing involves estimating all or some of:
the number of endmembers, their spectral signatures, and their abundances at
each pixel. Unmixing is a challenging, ill-posed inverse problem because of
model inaccuracies, observation noise, environmental conditions, endmember
variability, and data set size. Researchers have devised and investigated many
models searching for robust, stable, tractable, and accurate unmixing
algorithms. This paper presents an overview of unmixing methods from the time
of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models
are first discussed. Signal-subspace, geometrical, statistical, sparsity-based,
and spatial-contextual unmixing algorithms are described. Mathematical problems
and potential solutions are described. Algorithm characteristics are
illustrated experimentally.Comment: This work has been accepted for publication in IEEE Journal of
Selected Topics in Applied Earth Observations and Remote Sensin
Hyperspectral Image Unmixing Incorporating Adjacency Information
While the spectral information contained in hyperspectral images is rich, the spatial resolution of such images is in many cases very low. Many pixel spectra are mixtures of pure materials’ spectra and therefore need to be decomposed into their constituents. This work investigates new decomposition methods taking into account spectral, spatial and global 3D adjacency information. This allows for faster and more accurate decomposition results
- …