7 research outputs found
Learning Light Field Angular Super-Resolution via a Geometry-Aware Network
The acquisition of light field images with high angular resolution is costly.
Although many methods have been proposed to improve the angular resolution of a
sparsely-sampled light field, they always focus on the light field with a small
baseline, which is captured by a consumer light field camera. By making full
use of the intrinsic \textit{geometry} information of light fields, in this
paper we propose an end-to-end learning-based approach aiming at angularly
super-resolving a sparsely-sampled light field with a large baseline. Our model
consists of two learnable modules and a physically-based module. Specifically,
it includes a depth estimation module for explicitly modeling the scene
geometry, a physically-based warping for novel views synthesis, and a light
field blending module specifically designed for light field reconstruction.
Moreover, we introduce a novel loss function to promote the preservation of the
light field parallax structure. Experimental results over various light field
datasets including large baseline light field images demonstrate the
significant superiority of our method when compared with state-of-the-art ones,
i.e., our method improves the PSNR of the second best method up to 2 dB in
average, while saves the execution time 48. In addition, our method
preserves the light field parallax structure better.Comment: This paper was accepted by AAAI 202
Structure Modeling of High Dimensional Data:New Algorithms and Applications
The digitization of our most common appliances has led to a literal data deluge, some- times referred to as Big Data. The ever increasing volume of data we generate, coupled with our desire to exploit it ever faster, forces us to come up with innovative data pro- cessing techniques. Interestingly, the information we often look for has a very specific structure that distinguishes it from pure clutter. In this thesis, we explore the use of structured representations to propose new sensing techniques that severely limit the data throughput necessary to recover meaningful information. In particular, we exploit the intrinsic low-dimensionality of light field videos using tensor low-rank and sparse constraints to recover light field views from a single coded image per video frame. As opposed to conventional methods, our scheme neither alters the spatial resolution for angular resolution nor requires computationally extensive learning stage but rather depends on the intrinsic structures of light fields. In the second part of this thesis, we propose a novel algorithm to estimate depth from light fields. This method is based on representation of each patch in a light field view as a linear combination of patches from other views for a set of depth hypotheses. The structure in this representation is deployed to estimate accurate depth values. Finally, we introduce a low-power multi-channel cortical signal acquisition based on compressive sampling theory as an alternative to Nyquist-Shannon sampling theorem. Our scheme exploits the strong correlations between cortical signals to recover neural signals from a few compressive measurements
Momentum-Net: Fast and convergent iterative neural network for inverse problems
Iterative neural networks (INN) are rapidly gaining attention for solving
inverse problems in imaging, image processing, and computer vision. INNs
combine regression NNs and an iterative model-based image reconstruction (MBIR)
algorithm, often leading to both good generalization capability and
outperforming reconstruction quality over existing MBIR optimization models.
This paper proposes the first fast and convergent INN architecture,
Momentum-Net, by generalizing a block-wise MBIR algorithm that uses momentum
and majorizers with regression NNs. For fast MBIR, Momentum-Net uses momentum
terms in extrapolation modules, and noniterative MBIR modules at each iteration
by using majorizers, where each iteration of Momentum-Net consists of three
core modules: image refining, extrapolation, and MBIR. Momentum-Net guarantees
convergence to a fixed-point for general differentiable (non)convex MBIR
functions (or data-fit terms) and convex feasible sets, under two asymptomatic
conditions. To consider data-fit variations across training and testing
samples, we also propose a regularization parameter selection scheme based on
the "spectral spread" of majorization matrices. Numerical experiments for
light-field photography using a focal stack and sparse-view computational
tomography demonstrate that, given identical regression NN architectures,
Momentum-Net significantly improves MBIR speed and accuracy over several
existing INNs; it significantly improves reconstruction quality compared to a
state-of-the-art MBIR method in each application.Comment: 28 pages, 13 figures, 3 algorithms, 4 tables, submitted revision to
IEEE T-PAM
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Recommended from our members
Depth Estimation from a Single Holoscopic 3D Image and Image Up-sampling with Deep-learning
This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London3D depth information is widely utilized in industries such as security, autonomous vehicles, robotics, 3D printing, AR/VR entertainment, cinematography and medical science. However, state-of-the-art imaging and 3D depth-sensing technologies are rather complicated or expensive and still lack scalability and interoperability. The research identified, entails the development of an innovative technique for reliable and efficient 3D depth estimation that deliver better accuracy. The proposed (1) multilayer Holoscopic 3D encoding technique reduces the computational cost of extracting viewpoint images from complex structured Holoscopic 3D data by 95%, by using labelled multilayer elemental images. It also addresses misplacement of elemental image pixels due to lens distortion error. The multilayer Holoscopic 3D encoding computing efficiency leads to the implementation of real-time 3D depth-dependent applications. Also, (2) an innovative approach of a deep learning-based single image super-resolution framework is developed and evaluated. It identified that learning-based image up-sampling techniques could be used regardless of inadequate 3D training data, as 2D training data can yield the same results.
(3) The research is extended further by implementation of an H3D depth disparity -based framework, where a Holoscopic content adaptation technique for extracting semi-segmented stereo viewpoint image is introduced, and the design of a smart 3D depth mapping technique is proposed. Particularly, it provides a somewhat accurate 3D depth estimation from H3D images in near real-time. Holoscopic 3D image has thousands of perspective elemental images from omnidirectional viewpoint images and (4) a novel 3D depth estimation technique is developed to estimates 3D depth information directly from a single Holoscopic 3D image without the loss of any angular information and the introduction of unwanted artefacts. The proposed 3D depth measurement techniques are computationally efficient and robust with high accuracy; these can be incorporated in real-time applications of autonomous vehicles, security and AR/VR for real-time interaction
Tensor low-rank and sparse light field photography
High-quality light field photography has been one of the most difficult challenges in computational photography. Conventional methods either sacrifice resolution, use multiple devices, or require multiple images to be captured. Combining coded image acquisition and compressive reconstruction is one of the most promising directions to overcome limitations of conventional light field cameras. We present a new approach to compressive light field photography that exploits a joint tensor low-rank and sparse prior (LRSP) on natural light fields. As opposed to recently proposed light field dictionaries, our method does not require a computationally expensive learning stage but rather models the redundancies of high dimensional visual signals using a tensor low-rank prior. This is not only computationally more efficient but also more flexible in that the proposed techniques are easily applicable to a wide range of different imaging systems, camera parameters, and also scene types. (C) 2015 Elsevier Inc. All rights reserved