3 research outputs found
Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization
Light field (LF) images acquired by hand-held devices usually suffer from low
spatial resolution as the limited sampling resources have to be shared with the
angular dimension. LF spatial super-resolution (SR) thus becomes an
indispensable part of the LF camera processing pipeline. The
high-dimensionality characteristic and complex geometrical structure of LF
images make the problem more challenging than traditional single-image SR. The
performance of existing methods is still limited as they fail to thoroughly
explore the coherence among LF views and are insufficient in accurately
preserving the parallax structure of the scene. In this paper, we propose a
novel learning-based LF spatial SR framework, in which each view of an LF image
is first individually super-resolved by exploring the complementary information
among views with combinatorial geometry embedding. For accurate preservation of
the parallax structure among the reconstructed views, a regularization network
trained over a structure-aware loss function is subsequently appended to
enforce correct parallax relationships over the intermediate estimation. Our
proposed approach is evaluated over datasets with a large number of testing
images including both synthetic and real-world scenes. Experimental results
demonstrate the advantage of our approach over state-of-the-art methods, i.e.,
our method not only improves the average PSNR by more than 1.0 dB but also
preserves more accurate parallax details, at a lower computational cost.Comment: This paper was accepted by CVPR 202
Spatial-Angular Interaction for Light Field Image Super-Resolution
Light field (LF) cameras record both intensity and directions of light rays,
and capture scenes from a number of viewpoints. Both information within each
perspective (i.e., spatial information) and among different perspectives (i.e.,
angular information) is beneficial to image super-resolution (SR). In this
paper, we propose a spatial-angular interactive network (namely, LF-InterNet)
for LF image SR. Specifically, spatial and angular features are first
separately extracted from input LFs, and then repetitively interacted to
progressively incorporate spatial and angular information. Finally, the
interacted features are fused to superresolve each sub-aperture image.
Experimental results demonstrate the superiority of LF-InterNet over the
state-of-the-art methods, i.e., our method can achieve high PSNR and SSIM
scores with low computational cost, and recover faithful details in the
reconstructed images.Comment: In this version, we have revised the paper and compared our
LF-InterNet to the most recent LF-ATO method (CVPR2020). Codes and
pre-trained models are available at
https://github.com/YingqianWang/LF-InterNe
Light Field Image Super-Resolution Using Deformable Convolution
Light field (LF) cameras can record scenes from multiple perspectives, and
thus introduce beneficial angular information for image super-resolution (SR).
However, it is challenging to incorporate angular information due to
disparities among LF images. In this paper, we propose a deformable convolution
network (i.e., LF-DFnet) to handle the disparity problem for LF image SR.
Specifically, we design an angular deformable alignment module (ADAM) for
feature-level alignment. Based on ADAM, we further propose a
collect-and-distribute approach to perform bidirectional alignment between the
center-view feature and each side-view feature. Using our approach, angular
information can be well incorporated and encoded into features of each view,
which benefits the SR reconstruction of all LF images. Moreover, we develop a
baseline-adjustable LF dataset to evaluate SR performance under different
disparity variations. Experiments on both public and our self-developed
datasets have demonstrated the superiority of our method. Our LF-DFnet can
generate high-resolution images with more faithful details and achieve
state-of-the-art reconstruction accuracy. Besides, our LF-DFnet is more robust
to disparity variations, which has not been well addressed in literature.Comment: Accepted by IEEE Transactions on Image Processin