3,521 research outputs found
L1-norm Tucker Tensor Decomposition
Tucker decomposition is a common method for the analysis of multi-way/tensor
data. Standard Tucker has been shown to be sensitive against heavy corruptions,
due to its L2-norm-based formulation which places squared emphasis to
peripheral entries. In this work, we explore L1-Tucker, an L1-norm based
reformulation of standard Tucker decomposition. After formulating the problem,
we present two algorithms for its solution, namely L1-norm Higher-Order
Singular Value Decomposition (L1-HOSVD) and L1-norm Higher-Order Orthogonal
Iterations (L1-HOOI). The presented algorithms are accompanied by complexity
and convergence analysis. Our numerical studies on tensor reconstruction and
classification corroborate that L1-Tucker, implemented by means of the proposed
methods, attains similar performance to standard Tucker when the processed data
are corruption-free, while it exhibits sturdy resistance against heavily
corrupted entries
Robust Low-Rank Tensor Ring Completion
Low-rank tensor completion recovers missing entries based on different tensor
decompositions. Due to its outstanding performance in exploiting some
higher-order data structure, low rank tensor ring has been applied in tensor
completion. To further deal with its sensitivity to sparse component as it does
in tensor principle component analysis, we propose robust tensor ring
completion (RTRC), which separates latent low-rank tensor component from sparse
component with limited number of measurements. The low rank tensor component is
constrained by the weighted sum of nuclear norms of its balanced unfoldings,
while the sparse component is regularized by its l1 norm. We analyze the RTRC
model and gives the exact recovery guarantee. The alternating direction method
of multipliers is used to divide the problem into several sub-problems with
fast solutions. In numerical experiments, we verify the recovery condition of
the proposed method on synthetic data, and show the proposed method outperforms
the state-of-the-art ones in terms of both accuracy and computational
complexity in a number of real-world data based tasks, i.e., light-field image
recovery, shadow removal in face images, and background extraction in color
video
Bayesian Tensorized Neural Networks with Automatic Rank Selection
Tensor decomposition is an effective approach to compress over-parameterized
neural networks and to enable their deployment on resource-constrained hardware
platforms. However, directly applying tensor compression in the training
process is a challenging task due to the difficulty of choosing a proper tensor
rank. In order to achieve this goal, this paper proposes a Bayesian tensorized
neural network. Our Bayesian method performs automatic model compression via an
adaptive tensor rank determination. We also present approaches for posterior
density calculation and maximum a posteriori (MAP) estimation for the
end-to-end training of our tensorized neural network. We provide experimental
validation on a fully connected neural network, a CNN and a residual neural
network where our work produces to more compact neural
networks directly from the training
Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
The widespread use of multi-sensor technology and the emergence of big
datasets has highlighted the limitations of standard flat-view matrix models
and the necessity to move towards more versatile data analysis tools. We show
that higher-order tensors (i.e., multiway arrays) enable such a fundamental
paradigm shift towards models that are essentially polynomial and whose
uniqueness, unlike the matrix methods, is guaranteed under verymild and natural
conditions. Benefiting fromthe power ofmultilinear algebra as theirmathematical
backbone, data analysis techniques using tensor decompositions are shown to
have great flexibility in the choice of constraints that match data properties,
and to find more general latent components in the data than matrix-based
methods. A comprehensive introduction to tensor decompositions is provided from
a signal processing perspective, starting from the algebraic foundations, via
basic Canonical Polyadic and Tucker models, through to advanced cause-effect
and multi-view data analysis schemes. We show that tensor decompositions enable
natural generalizations of some commonly used signal processing paradigms, such
as canonical correlation and subspace techniques, signal separation, linear
regression, feature extraction and classification. We also cover computational
aspects, and point out how ideas from compressed sensing and scientific
computing may be used for addressing the otherwise unmanageable storage and
manipulation problems associated with big datasets. The concepts are supported
by illustrative real world case studies illuminating the benefits of the tensor
framework, as efficient and promising tools for modern signal processing, data
analysis and machine learning applications; these benefits also extend to
vector/matrix data through tensorization. Keywords: ICA, NMF, CPD, Tucker
decomposition, HOSVD, tensor networks, Tensor Train
Tensor train-Karhunen-Lo\`eve expansion for continuous-indexed random fields using higher-order cumulant functions
The goals of this work are two-fold: firstly, to propose a new theoretical
framework for representing random fields on a large class of multidimensional
geometrical domain in the tensor train format; secondly, to develop a new
algorithm framework for accurately computing the modes and the second and
third-order cumulant tensors within moderate time. The core of the new
theoretical framework is the tensor train decomposition of cumulant functions.
This decomposition is accurately computed with a novel rank-revealing
algorithm. Compared with existing Galerkin-type and collocation-type methods,
the proposed computational procedure totally removes the need of selecting the
basis functions or collocation points and the quadrature points, which not only
greatly enhances adaptivity, but also avoids solving large-scale eigenvalue
problems. Moreover, by computing with third-order cumulant functions, the new
theoretical and algorithm frameworks show great potential for representing
general non-Gaussian non-homogeneous random fields. Three numerical examples,
including a three-dimensional random field discretization problem, illustrate
the efficiency and accuracy of the proposed algorithm framework
Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems
In this paper we review basic and emerging models and associated algorithms
for large-scale tensor networks, especially Tensor Train (TT) decompositions
using novel mathematical and graphical representations. We discus the concept
of tensorization (i.e., creating very high-order tensors from lower-order
original data) and super compression of data achieved via quantized tensor
train (QTT) networks. The purpose of a tensorization and quantization is to
achieve, via low-rank tensor approximations "super" compression, and
meaningful, compact representation of structured data. The main objective of
this paper is to show how tensor networks can be used to solve a wide class of
big data optimization problems (that are far from tractable by classical
numerical methods) by applying tensorization and performing all operations
using relatively small size matrices and tensors and applying iteratively
optimized and approximative tensor contractions.
Keywords: Tensor networks, tensor train (TT) decompositions, matrix product
states (MPS), matrix product operators (MPO), basic tensor operations,
tensorization, distributed representation od data optimization problems for
very large-scale problems: generalized eigenvalue decomposition (GEVD),
PCA/SVD, canonical correlation analysis (CCA).Comment: arXiv admin note: text overlap with arXiv:1403.204
Sample, computation vs storage tradeoffs for classification using tensor subspace models
In this paper, we exhibit the tradeoffs between the (training) sample,
computation and storage complexity for the problem of supervised classification
using signal subspace estimation. Our main tool is the use of tensor subspaces,
i.e. subspaces with a Kronecker structure, for embedding the data into lower
dimensions. Among the subspaces with a Kronecker structure, we show that using
subspaces with a hierarchical structure for representing data leads to improved
tradeoffs. One of the main reasons for the improvement is that embedding data
into these hierarchical Kronecker structured subspaces prevents overfitting at
higher latent dimensions.Comment: 5 Page
An optimization approach for dynamical Tucker tensor approximation
An optimization-based approach for the Tucker tensor approximation of
parameter-dependent data tensors and solutions of tensor differential equations
with low Tucker rank is presented. The problem of updating the tensor
decomposition is reformulated as fitting problem subject to the tangent space
without relying on an orthogonality gauge condition. A discrete Euler scheme is
established in an alternating least squares framework, where the quadratic
subproblems reduce to trace optimization problems, that are shown to be
explicitly solvable and accessible using SVD of small size. In the presence of
small singular values, instability for larger ranks is reduced, since the
method does not need the (pseudo) inverse of matricizations of the core tensor.
Regularization of Tikhonov type can be used to compensate for the lack of
uniqueness in the tangent space. The method is validated numerically and shown
to be stable also for larger ranks in the case of small singular values of the
core unfoldings. Higher order explicit integrators of Runge-Kutta type can be
composed.Comment: 18 pages, 10 figure
Non-recurrent Traffic Congestion Detection with a Coupled Scalable Bayesian Robust Tensor Factorization Model
Non-recurrent traffic congestion (NRTC) usually brings unexpected delays to
commuters. Hence, it is critical to accurately detect and recognize the NRTC in
a real-time manner. The advancement of road traffic detectors and loop
detectors provides researchers with a large-scale multivariable
temporal-spatial traffic data, which allows the deep research on NRTC to be
conducted. However, it remains a challenging task to construct an analytical
framework through which the natural spatial-temporal structural properties of
multivariable traffic information can be effectively represented and exploited
to better understand and detect NRTC. In this paper, we present a novel
analytical training-free framework based on coupled scalable Bayesian robust
tensor factorization (Coupled SBRTF). The framework can couple multivariable
traffic data including traffic flow, road speed, and occupancy through sharing
a similar or the same sparse structure. And, it naturally captures the
high-dimensional spatial-temporal structural properties of traffic data by
tensor factorization. With its entries revealing the distribution and magnitude
of NRTC, the shared sparse structure of the framework compasses sufficiently
abundant information about NRTC. While the low-rank part of the framework,
expresses the distribution of general expected traffic condition as an
auxiliary product. Experimental results on real-world traffic data show that
the proposed method outperforms coupled Bayesian robust principal component
analysis (coupled BRPCA), the rank sparsity tensor decomposition (RSTD), and
standard normal deviates (SND) in detecting NRTC. The proposed method performs
even better when only traffic data in weekdays are utilized, and hence can
provide more precise estimation of NRTC for daily commuters
Vectorial Dimension Reduction for Tensors Based on Bayesian Inference
Dimensionality reduction for high-order tensors is a challenging problem. In
conventional approaches, higher order tensors are `vectorized` via Tucker
decomposition to obtain lower order tensors. This will destroy the inherent
high-order structures or resulting in undesired tensors, respectively. This
paper introduces a probabilistic vectorial dimensionality reduction model for
tensorial data. The model represents a tensor by employing a linear combination
of same order basis tensors, thus it offers a mechanism to directly reduce a
tensor to a vector. Under this expression, the projection base of the model is
based on the tensor CandeComp/PARAFAC (CP) decomposition and the number of free
parameters in the model only grows linearly with the number of modes rather
than exponentially. A Bayesian inference has been established via the
variational EM approach. A criterion to set the parameters (factor number of CP
decomposition and the number of extracted features) is empirically given. The
model outperforms several existing PCA-based methods and CP decomposition on
several publicly available databases in terms of classification and clustering
accuracy.Comment: Submiting to TNNL
- …