605 research outputs found
High-quality hyperspectral reconstruction using a spectral prior
We present a novel hyperspectral image reconstruction algorithm, which overcomes the long-standing tradeoff between spectral accuracy and spatial resolution in existing compressive imaging approaches. Our method consists of two steps: First, we learn nonlinear spectral representations from real-world hyperspectral datasets; for this, we build a convolutional autoencoder, which allows reconstructing its own input through its encoder and decoder networks. Second, we introduce a novel optimization method, which jointly regularizes the fidelity of the learned nonlinear spectral representations and the sparsity of gradients in the spatial domain, by means of our new fidelity prior. Our technique can be applied to any existing compressive imaging architecture, and has been thoroughly tested both in simulation, and by building a prototype hyperspectral imaging system. It outperforms the state-of-the-art methods from each architecture, both in terms of spectral accuracy and spatial resolution, while its computational complexity is reduced by two orders of magnitude with respect to sparse coding techniques. Moreover, we present two additional applications of our method: hyperspectral interpolation and demosaicing. Last, we have created a new high-resolution hyperspectral dataset containing sharper images of more spectral variety than existing ones, available through our project website
Generalized Inpainting Method for Hyperspectral Image Acquisition
A recently designed hyperspectral imaging device enables multiplexed
acquisition of an entire data volume in a single snapshot thanks to
monolithically-integrated spectral filters. Such an agile imaging technique
comes at the cost of a reduced spatial resolution and the need for a
demosaicing procedure on its interleaved data. In this work, we address both
issues and propose an approach inspired by recent developments in compressed
sensing and analysis sparse models. We formulate our superresolution and
demosaicing task as a 3-D generalized inpainting problem. Interestingly, the
target spatial resolution can be adjusted for mitigating the compression level
of our sensing. The reconstruction procedure uses a fast greedy method called
Pseudo-inverse IHT. We also show on simulations that a random arrangement of
the spectral filters on the sensor is preferable to regular mosaic layout as it
improves the quality of the reconstruction. The efficiency of our technique is
demonstrated through numerical experiments on both synthetic and real data as
acquired by the snapshot imager.Comment: Keywords: Hyperspectral, inpainting, iterative hard thresholding,
sparse models, CMOS, Fabry-P\'ero
S^2-Transformer for Mask-Aware Hyperspectral Image Reconstruction
The technology of hyperspectral imaging (HSI) records the visual information
upon long-range-distributed spectral wavelengths. A representative
hyperspectral image acquisition procedure conducts a 3D-to-2D encoding by the
coded aperture snapshot spectral imager (CASSI) and requires a software decoder
for the 3D signal reconstruction. By observing this physical encoding
procedure, two major challenges stand in the way of a high-fidelity
reconstruction. (i) To obtain 2D measurements, CASSI dislocates multiple
channels by disperser-titling and squeezes them onto the same spatial region,
yielding an entangled data loss. (ii) The physical coded aperture leads to a
masked data loss by selectively blocking the pixel-wise light exposure. To
tackle these challenges, we propose a spatial-spectral (S^2-) Transformer
network with a mask-aware learning strategy. First, we simultaneously leverage
spatial and spectral attention modeling to disentangle the blended information
in the 2D measurement along both two dimensions. A series of Transformer
structures are systematically designed to fully investigate the spatial and
spectral informative properties of the hyperspectral data. Second, the masked
pixels will induce higher prediction difficulty and should be treated
differently from unmasked ones. Thereby, we adaptively prioritize the loss
penalty attributing to the mask structure by inferring the pixel-wise
reconstruction difficulty upon the mask-encoded prediction. We theoretically
discusses the distinct convergence tendencies between masked/unmasked regions
of the proposed learning strategy. Extensive experiments demonstrates that the
proposed method achieves superior reconstruction performance. Additionally, we
empirically elaborate the behaviour of spatial and spectral attentions under
the proposed architecture, and comprehensively examine the impact of the
mask-aware learning.Comment: 11 pages, 16 figures, 6 tables, Code:
https://github.com/Jiamian-Wang/S2-transformer-HS
- …