7 research outputs found
Multispectral snapshot demosaicing via non-convex matrix completion
Snapshot mosaic multispectral imagery acquires an undersampled data cube by
acquiring a single spectral measurement per spatial pixel. Sensors which
acquire frequencies, therefore, suffer from severe undersampling of
the full data cube. We show that the missing entries can be accurately imputed
using non-convex techniques from sparse approximation and matrix completion
initialised with traditional demosaicing algorithms. In particular, we observe
the peak signal-to-noise ratio can typically be improved by 2 to 5 dB over
current state-of-the-art methods when simulating a mosaic sensor
measuring both high and low altitude urban and rural scenes as well as
ground-based scenes.Comment: 5 pages, 2 figures, 1 tabl
Multispectral Stereo-Image Fusion for 3D Hyperspectral Scene Reconstruction
Spectral imaging enables the analysis of optical material properties that are
invisible to the human eye. Different spectral capturing setups, e.g., based on
filter-wheel, push-broom, line-scanning, or mosaic cameras, have been
introduced in the last years to support a wide range of applications in
agriculture, medicine, and industrial surveillance. However, these systems
often suffer from different disadvantages, such as lack of real-time
capability, limited spectral coverage or low spatial resolution. To address
these drawbacks, we present a novel approach combining two calibrated
multispectral real-time capable snapshot cameras, covering different spectral
ranges, into a stereo-system. Therefore, a hyperspectral data-cube can be
continuously captured. The combined use of different multispectral snapshot
cameras enables both 3D reconstruction and spectral analysis. Both captured
images are demosaicked avoiding spatial resolution loss. We fuse the spectral
data from one camera into the other to receive a spatially and spectrally high
resolution video stream. Experiments demonstrate the feasibility of this
approach and the system is investigated with regard to its applicability for
surgical assistance monitoring.Comment: VISAPP 2024 - 19th International Conference on Computer Vision Theory
and Application
Aperture Diffraction for Compact Snapshot Spectral Imaging
We demonstrate a compact, cost-effective snapshot spectral imaging system
named Aperture Diffraction Imaging Spectrometer (ADIS), which consists only of
an imaging lens with an ultra-thin orthogonal aperture mask and a mosaic filter
sensor, requiring no additional physical footprint compared to common RGB
cameras. Then we introduce a new optical design that each point in the object
space is multiplexed to discrete encoding locations on the mosaic filter sensor
by diffraction-based spatial-spectral projection engineering generated from the
orthogonal mask. The orthogonal projection is uniformly accepted to obtain a
weakly calibration-dependent data form to enhance modulation robustness.
Meanwhile, the Cascade Shift-Shuffle Spectral Transformer (CSST) with strong
perception of the diffraction degeneration is designed to solve a
sparsity-constrained inverse problem, realizing the volume reconstruction from
2D measurements with Large amount of aliasing. Our system is evaluated by
elaborating the imaging optical theory and reconstruction algorithm with
demonstrating the experimental imaging under a single exposure. Ultimately, we
achieve the sub-super-pixel spatial resolution and high spectral resolution
imaging. The code will be available at: https://github.com/Krito-ex/CSST.Comment: accepted by International Conference on Computer Vision (ICCV) 202
InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments
Classic designs of hyperspectral instrumentation densely sample the spatial
and spectral information of the scene of interest. Data may be compressed after
the acquisition. In this paper we introduce a framework for the design of an
optimized, micro-patterned snapshot hyperspectral imager that acquires an
optimized subset of the spatial and spectral information in the scene. The data
is thereby compressed already at the sensor level, but can be restored to the
full hyperspectral data cube by the jointly optimized reconstructor. This
framework is implemented with TensorFlow and makes use of its automatic
differentiation for the joint optimization of the layout of the micro-patterned
filter array as well as the reconstructor. We explore the achievable
compression ratio for different numbers of filter passbands, number of scanning
frames, and filter layouts using data collected by the Hyperscout instrument.
We show resulting instrument designs that take snapshot measurements without
losing significant information while reducing the data volume, acquisition
time, or detector space by a factor of 40 as compared to classic, dense
sampling. The joint optimization of a compressive hyperspectral imager design
and the accompanying reconstructor provides an avenue to substantially reduce
the data volume from hyperspectral imagers.Comment: 23 pages, 12 figures, published in Applied Optic
Multispectral demosaicing using pseudo-panchromatic image
International audienceSingle-sensor color cameras, which classically use a color filter array (CFA) to sample RGB channels, have recently been extended to the multispectral domain. To sample more than three wavelength bands, such systems use a multispectral filter array (MSFA) that provides a raw image in which a single channel value is available at each pixel. A demosaicing procedure is then needed to estimate a fully-defined multispectral image. In this paper, we review multispectral demosaicing methods and propose a new one based on the pseudo-panchromatic image (PPI). Pixel values in the PPI are computed as the average spectral values. Experimental results show that our method provides estimated images of better quality than classical ones