134 research outputs found
A weighted subspace exponential kernel for support tensor machines
High-dimensional data in the form of tensors are challenging for kernel
classification methods. To both reduce the computational complexity and extract
informative features, kernels based on low-rank tensor decompositions have been
proposed. However, what decisive features of the tensors are exploited by these
kernels is often unclear. In this paper we propose a novel kernel that is based
on the Tucker decomposition. For this kernel the Tucker factors are computed
based on re-weighting of the Tucker matrices with tuneable powers of singular
values from the HOSVD decomposition. This provides a mechanism to balance the
contribution of the Tucker core and factors of the data. We benchmark support
tensor machines with this new kernel on several datasets. First we generate
synthetic data where two classes differ in either Tucker factors or core, and
compare our novel and previously existing kernels. We show robustness of the
new kernel with respect to both classification scenarios. We further test the
new method on real-world datasets. The proposed kernel has demonstrated a
higher test accuracy than the state-of-the-art tensor train multi-way
multi-level kernel, and a significantly lower computational time
GloptiNets: Scalable Non-Convex Optimization with Certificates
We present a novel approach to non-convex optimization with certificates,
which handles smooth functions on the hypercube or on the torus. Unlike
traditional methods that rely on algebraic properties, our algorithm exploits
the regularity of the target function intrinsic in the decay of its Fourier
spectrum. By defining a tractable family of models, we allow at the same time
to obtain precise certificates and to leverage the advanced and powerful
computational techniques developed to optimize neural networks. In this way the
scalability of our approach is naturally enhanced by parallel computing with
GPUs. Our approach, when applied to the case of polynomials of moderate
dimensions but with thousands of coefficients, outperforms the state-of-the-art
optimization methods with certificates, as the ones based on Lasserre's
hierarchy, addressing problems intractable for the competitors
Rank-preserving geometric means of positive semi-definite matrices
The generalization of the geometric mean of positive scalars to positive
definite matrices has attracted considerable attention since the seminal work
of Ando. The paper generalizes this framework of matrix means by proposing the
definition of a rank-preserving mean for two or an arbitrary number of positive
semi-definite matrices of fixed rank. The proposed mean is shown to be
geometric in that it satisfies all the expected properties of a rank-preserving
geometric mean. The work is motivated by operations on low-rank approximations
of positive definite matrices in high-dimensional spaces.Comment: To appear in Linear Algebra and its Application
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds
Sparsity-based representations have recently led to notable results in
various visual recognition tasks. In a separate line of research, Riemannian
manifolds have been shown useful for dealing with features and models that do
not lie in Euclidean spaces. With the aim of building a bridge between the two
realms, we address the problem of sparse coding and dictionary learning over
the space of linear subspaces, which form Riemannian structures known as
Grassmann manifolds. To this end, we propose to embed Grassmann manifolds into
the space of symmetric matrices by an isometric mapping. This in turn enables
us to extend two sparse coding schemes to Grassmann manifolds. Furthermore, we
propose closed-form solutions for learning a Grassmann dictionary, atom by
atom. Lastly, to handle non-linearity in data, we extend the proposed Grassmann
sparse coding and dictionary learning algorithms through embedding into Hilbert
spaces.
Experiments on several classification tasks (gender recognition, gesture
classification, scene analysis, face recognition, action recognition and
dynamic texture classification) show that the proposed approaches achieve
considerable improvements in discrimination accuracy, in comparison to
state-of-the-art methods such as kernelized Affine Hull Method and
graph-embedding Grassmann discriminant analysis.Comment: Appearing in International Journal of Computer Visio
Moment methods in energy minimization: New bounds for Riesz minimal energy problems
We use moment methods to construct a converging hierarchy of optimization
problems to lower bound the ground state energy of interacting particle
systems. We approximate the infinite dimensional optimization problems in this
hierarchy by block diagonal semidefinite programs. For this we develop the
necessary harmonic analysis for spaces consisting of subsets of another space,
and we develop symmetric sum-of-squares techniques. We compute the second step
of our hierarchy for Riesz -energy problems with five particles on the
-dimensional unit sphere, where the case known as the Thomson problem.
This yields new sharp bounds (up to high precision) and suggests the second
step of our hierarchy may be sharp throughout a phase transition and may be
universally sharp for -particles on . This is the first time a
-point bound has been computed for a continuous problem
- …