8,803 research outputs found
Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data
With the increasing availability of various sensor technologies, we now have
access to large amounts of multi-block (also called multi-set,
multi-relational, or multi-view) data that need to be jointly analyzed to
explore their latent connections. Various component analysis methods have
played an increasingly important role for the analysis of such coupled data. In
this paper, we first provide a brief review of existing matrix-based (two-way)
component analysis methods for the joint analysis of such data with a focus on
biomedical applications. Then, we discuss their important extensions and
generalization to multi-block multiway (tensor) data. We show how constrained
multi-block tensor decomposition methods are able to extract similar or
statistically dependent common features that are shared by all blocks, by
incorporating the multiway nature of data. Special emphasis is given to the
flexible common and individual feature analysis of multi-block data with the
aim to simultaneously extract common and individual latent components with
desired properties and types of diversity. Illustrative examples are given to
demonstrate their effectiveness for biomedical data analysis.Comment: 20 pages, 11 figures, Proceedings of the IEEE, 201
An Iterative Reweighted Method for Tucker Decomposition of Incomplete Multiway Tensors
We consider the problem of low-rank decomposition of incomplete multiway
tensors. Since many real-world data lie on an intrinsically low dimensional
subspace, tensor low-rank decomposition with missing entries has applications
in many data analysis problems such as recommender systems and image
inpainting. In this paper, we focus on Tucker decomposition which represents an
Nth-order tensor in terms of N factor matrices and a core tensor via
multilinear operations. To exploit the underlying multilinear low-rank
structure in high-dimensional datasets, we propose a group-based log-sum
penalty functional to place structural sparsity over the core tensor, which
leads to a compact representation with smallest core tensor. The method for
Tucker decomposition is developed by iteratively minimizing a surrogate
function that majorizes the original objective function, which results in an
iterative reweighted process. In addition, to reduce the computational
complexity, an over-relaxed monotone fast iterative shrinkage-thresholding
technique is adapted and embedded in the iterative reweighted process. The
proposed method is able to determine the model complexity (i.e. multilinear
rank) in an automatic way. Simulation results show that the proposed algorithm
offers competitive performance compared with other existing algorithms
Tensor Decomposition for Signal Processing and Machine Learning
Tensors or {\em multi-way arrays} are functions of three or more indices
-- similar to matrices (two-way arrays), which are functions
of two indices for (row,column). Tensors have a rich history,
stretching over almost a century, and touching upon numerous disciplines; but
they have only recently become ubiquitous in signal and data analytics at the
confluence of signal processing, statistics, data mining and machine learning.
This overview article aims to provide a good starting point for researchers and
practitioners interested in learning about and working with tensors. As such,
it focuses on fundamentals and motivation (using various application examples),
aiming to strike an appropriate balance of breadth {\em and depth} that will
enable someone having taken first graduate courses in matrix algebra and
probability to get started doing research and/or developing tensor algorithms
and software. Some background in applied optimization is useful but not
strictly required. The material covered includes tensor rank and rank
decomposition; basic tensor factorization models and their relationships and
properties (including fairly good coverage of identifiability); broad coverage
of algorithms ranging from alternating optimization to stochastic gradient;
statistical performance analysis; and applications ranging from source
separation to collaborative filtering, mixture and topic modeling,
classification, and multilinear subspace learning.Comment: revised version, overview articl
Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
The widespread use of multi-sensor technology and the emergence of big
datasets has highlighted the limitations of standard flat-view matrix models
and the necessity to move towards more versatile data analysis tools. We show
that higher-order tensors (i.e., multiway arrays) enable such a fundamental
paradigm shift towards models that are essentially polynomial and whose
uniqueness, unlike the matrix methods, is guaranteed under verymild and natural
conditions. Benefiting fromthe power ofmultilinear algebra as theirmathematical
backbone, data analysis techniques using tensor decompositions are shown to
have great flexibility in the choice of constraints that match data properties,
and to find more general latent components in the data than matrix-based
methods. A comprehensive introduction to tensor decompositions is provided from
a signal processing perspective, starting from the algebraic foundations, via
basic Canonical Polyadic and Tucker models, through to advanced cause-effect
and multi-view data analysis schemes. We show that tensor decompositions enable
natural generalizations of some commonly used signal processing paradigms, such
as canonical correlation and subspace techniques, signal separation, linear
regression, feature extraction and classification. We also cover computational
aspects, and point out how ideas from compressed sensing and scientific
computing may be used for addressing the otherwise unmanageable storage and
manipulation problems associated with big datasets. The concepts are supported
by illustrative real world case studies illuminating the benefits of the tensor
framework, as efficient and promising tools for modern signal processing, data
analysis and machine learning applications; these benefits also extend to
vector/matrix data through tensorization. Keywords: ICA, NMF, CPD, Tucker
decomposition, HOSVD, tensor networks, Tensor Train
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor B-Spline Numerical Methods for PDEs: a High-Performance Alternative to FEM
Tensor B-spline methods are a high-performance alternative to solve partial
differential equations (PDEs). This paper gives an overview on the principles
of Tensor B-spline methodology, shows their use and analyzes their performance
in application examples, and discusses its merits. Tensors preserve the
dimensional structure of a discretized PDE, which makes it possible to develop
highly efficient computational solvers. B-splines provide high-quality
approximations, lead to a sparse structure of the system operator represented
by shift-invariant separable kernels in the domain, and are mesh-free by
construction. Further, high-order bases can easily be constructed from
B-splines. In order to demonstrate the advantageous numerical performance of
tensor B-spline methods, we studied the solution of a large-scale heat-equation
problem (consisting of roughly 0.8 billion nodes!) on a heterogeneous
workstation consisting of multi-core CPU and GPUs. Our experimental results
nicely confirm the excellent numerical approximation properties of tensor
B-splines, and their unique combination of high computational efficiency and
low memory consumption, thereby showing huge improvements over standard
finite-element methods (FEM)
Tensor Analysis and Fusion of Multimodal Brain Images
Current high-throughput data acquisition technologies probe dynamical systems
with different imaging modalities, generating massive data sets at different
spatial and temporal resolutions posing challenging problems in multimodal data
fusion. A case in point is the attempt to parse out the brain structures and
networks that underpin human cognitive processes by analysis of different
neuroimaging modalities (functional MRI, EEG, NIRS etc.). We emphasize that the
multimodal, multi-scale nature of neuroimaging data is well reflected by a
multi-way (tensor) structure where the underlying processes can be summarized
by a relatively small number of components or "atoms". We introduce
Markov-Penrose diagrams - an integration of Bayesian DAG and tensor network
notation in order to analyze these models. These diagrams not only clarify
matrix and tensor EEG and fMRI time/frequency analysis and inverse problems,
but also help understand multimodal fusion via Multiway Partial Least Squares
and Coupled Matrix-Tensor Factorization. We show here, for the first time, that
Granger causal analysis of brain networks is a tensor regression problem, thus
allowing the atomic decomposition of brain networks. Analysis of EEG and fMRI
recordings shows the potential of the methods and suggests their use in other
scientific domains.Comment: 23 pages, 15 figures, submitted to Proceedings of the IEE
Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition
Tensor network decomposition, originated from quantum physics to model
entangled many-particle quantum systems, turns out to be a promising
mathematical technique to efficiently represent and process big data in
parsimonious manner. In this study, we show that tensor networks can
systematically partition structured data, e.g. color images, for distributed
storage and communication in privacy-preserving manner. Leveraging the sea of
big data and metadata privacy, empirical results show that neighbouring
subtensors with implicit information stored in tensor network formats cannot be
identified for data reconstruction. This technique complements the existing
encryption and randomization techniques which store explicit data
representation at one place and highly susceptible to adversarial attacks such
as side-channel attacks and de-anonymization. Furthermore, we propose a theory
for adversarial examples that mislead convolutional neural networks to
misclassification using subspace analysis based on singular value decomposition
(SVD). The theory is extended to analyze higher-order tensors using
tensor-train SVD (TT-SVD); it helps to explain the level of susceptibility of
different datasets to adversarial attacks, the structural similarity of
different adversarial attacks including global and localized attacks, and the
efficacy of different adversarial defenses based on input transformation. An
efficient and adaptive algorithm based on robust TT-SVD is then developed to
detect strong and static adversarial attacks
Spectral Compressed Sensing via CANDECOMP/PARAFAC Decomposition of Incomplete Tensors
We consider the line spectral estimation problem which aims to recover a
mixture of complex sinusoids from a small number of randomly observed time
domain samples. Compressed sensing methods formulates line spectral estimation
as a sparse signal recovery problem by discretizing the continuous frequency
parameter space into a finite set of grid points. Discretization, however,
inevitably incurs errors and leads to deteriorated estimation performance. In
this paper, we propose a new method which leverages recent advances in tensor
decomposition. Specifically, we organize the observed data into a structured
tensor and cast line spectral estimation as a CANDECOMP/PARAFAC (CP)
decomposition problem with missing entries. The uniqueness of the CP
decomposition allows the frequency components to be super-resolved with
infinite precision. Simulation results show that the proposed method provides a
competitive estimate accuracy compared with existing state-of-the-art
algorithms
Canonical forms of Order- () Symmetric Tensors of Format Over Prime Fields
We consider symmetric tensors of format: over for
; over for ; and over for . In each case we
compute their equivalence classes under the action of the general linear group
. We use computer algebra to determine the set of tensors
of each symmetric rank, then we compute the orbit of the group action. We
determine the maximum symmetric rank of these tensors and compare it with the
maximum rank.Comment: 14 pages, 6 tables. arXiv admin note: text overlap with
arXiv:1209.502
- …