347 research outputs found
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Exploitation of source nonstationarity in underdetermined blind source separation with advanced clustering techniques
The problem of blind source separation (BSS) is
investigated. Following the assumption that the time-frequency
(TF) distributions of the input sources do not overlap, quadratic
TF representation is used to exploit the sparsity of the statistically
nonstationary sources. However, separation performance is shown
to be limited by the selection of a certain threshold in classifying
the eigenvectors of the TF matrices drawn from the observation
mixtures. Two methods are, therefore, proposed based on recently
introduced advanced clustering techniques, namely Gap statistics
and self-splitting competitive learning (SSCL), to mitigate the
problem of eigenvector classification. The novel integration of
these two approaches successfully overcomes the problem of artificial
sources induced by insufficient knowledge of the threshold and
enables automatic determination of the number of active sources
over the observation. The separation performance is thereby
greatly improved. Practical consequences of violating the TF orthogonality
assumption in the current approach are also studied,
which motivates the proposal of a new solution robust to violation
of orthogonality. In this new method, the TF plane is partitioned
into appropriate blocks and source separation is thereby carried
out in a block-by-block manner. Numerical experiments with
linear chirp signals and Gaussian minimum shift keying (GMSK)
signals are included which support the improved performance of
the proposed approaches
- …