283 research outputs found
Tensor Computation: A New Framework for High-Dimensional Problems in EDA
Many critical EDA problems suffer from the curse of dimensionality, i.e. the
very fast-scaling computational burden produced by large number of parameters
and/or unknown variables. This phenomenon may be caused by multiple spatial or
temporal factors (e.g. 3-D field solvers discretizations and multi-rate circuit
simulation), nonlinearity of devices and circuits, large number of design or
optimization parameters (e.g. full-chip routing/placement and circuit sizing),
or extensive process variations (e.g. variability/reliability analysis and
design for manufacturability). The computational challenges generated by such
high dimensional problems are generally hard to handle efficiently with
traditional EDA core algorithms that are based on matrix and vector
computation. This paper presents "tensor computation" as an alternative general
framework for the development of efficient EDA algorithms and tools. A tensor
is a high-dimensional generalization of a matrix and a vector, and is a natural
choice for both storing and solving efficiently high-dimensional EDA problems.
This paper gives a basic tutorial on tensors, demonstrates some recent examples
of EDA applications (e.g., nonlinear circuit modeling and high-dimensional
uncertainty quantification), and suggests further open EDA problems where the
use of tensor computation could be of advantage.Comment: 14 figures. Accepted by IEEE Trans. CAD of Integrated Circuits and
System
Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective
The proposed article aims at offering a comprehensive tutorial for the
computational aspects of structured matrix and tensor factorization. Unlike
existing tutorials that mainly focus on {\it algorithmic procedures} for a
small set of problems, e.g., nonnegativity or sparsity-constrained
factorization, we take a {\it top-down} approach: we start with general
optimization theory (e.g., inexact and accelerated block coordinate descent,
stochastic optimization, and Gauss-Newton methods) that covers a wide range of
factorization problems with diverse constraints and regularization terms of
engineering interest. Then, we go `under the hood' to showcase specific
algorithm design under these introduced principles. We pay a particular
attention to recent algorithmic developments in structured tensor and matrix
factorization (e.g., random sketching and adaptive step size based stochastic
optimization and structure-exploiting second-order algorithms), which are the
state of the art---yet much less touched upon in the literature compared to
{\it block coordinate descent} (BCD)-based methods. We expect that the article
to have an educational values in the field of structured factorization and hope
to stimulate more research in this important and exciting direction.Comment: Final Version; to appear in IEEE Signal Processing Magazine; title
revised to comply with the journal's rul
The average condition number of most tensor rank decomposition problems is infinite
The tensor rank decomposition, or canonical polyadic decomposition, is the
decomposition of a tensor into a sum of rank-1 tensors. The condition number of
the tensor rank decomposition measures the sensitivity of the rank-1 summands
with respect to structured perturbations. Those are perturbations preserving
the rank of the tensor that is decomposed. On the other hand, the angular
condition number measures the perturbations of the rank-1 summands up to
scaling.
We show for random rank-2 tensors with Gaussian density that the expected
value of the condition number is infinite. Under some mild additional
assumption, we show that the same is true for most higher ranks as
well. In fact, as the dimensions of the tensor tend to infinity, asymptotically
all ranks are covered by our analysis. On the contrary, we show that rank-2
Gaussian tensors have finite expected angular condition number.
Our results underline the high computational complexity of computing tensor
rank decompositions. We discuss consequences of our results for algorithm
design and for testing algorithms that compute the CPD. Finally, we supply
numerical experiments
Application Performance Modeling via Tensor Completion
Performance tuning, software/hardware co-design, and job scheduling are among
the many tasks that rely on models to predict application performance. We
propose and evaluate low-rank tensor decomposition for modeling application
performance. We discretize the input and configuration domains of an
application using regular grids. Application execution times mapped within
grid-cells are averaged and represented by tensor elements. We show that
low-rank canonical-polyadic (CP) tensor decomposition is effective in
approximating these tensors. We further show that this decomposition enables
accurate extrapolation of unobserved regions of an application's parameter
space. We then employ tensor completion to optimize a CP decomposition given a
sparse set of observed execution times. We consider alternative
piecewise/grid-based models and supervised learning models for six applications
and demonstrate that CP decomposition optimized using tensor completion offers
higher prediction accuracy and memory-efficiency for high-dimensional
performance modeling
Parallel Algorithms for Constrained Tensor Factorization via the Alternating Direction Method of Multipliers
Tensor factorization has proven useful in a wide range of applications, from
sensor array processing to communications, speech and audio signal processing,
and machine learning. With few recent exceptions, all tensor factorization
algorithms were originally developed for centralized, in-memory computation on
a single machine; and the few that break away from this mold do not easily
incorporate practically important constraints, such as nonnegativity. A new
constrained tensor factorization framework is proposed in this paper, building
upon the Alternating Direction method of Multipliers (ADMoM). It is shown that
this simplifies computations, bypassing the need to solve constrained
optimization problems in each iteration; and it naturally leads to distributed
algorithms suitable for parallel implementation on regular high-performance
computing (e.g., mesh) architectures. This opens the door for many emerging big
data-enabled applications. The methodology is exemplified using nonnegativity
as a baseline constraint, but the proposed framework can more-or-less readily
incorporate many other types of constraints. Numerical experiments are very
encouraging, indicating that the ADMoM-based nonnegative tensor factorization
(NTF) has high potential as an alternative to state-of-the-art approaches.Comment: Submitted to the IEEE Transactions on Signal Processin
- …