1,771 research outputs found
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Towards Dual-functional Radar-Communication Systems: Optimal Waveform Design
We focus on a dual-functional multi-input-multi-output (MIMO)
radar-communication (RadCom) system, where a single transmitter communicates
with downlink cellular users and detects radar targets simultaneously. Several
design criteria are considered for minimizing the downlink multi-user
interference. First, we consider both the omnidirectional and directional
beampattern design problems, where the closed-form globally optimal solutions
are obtained. Based on these waveforms, we further consider a weighted
optimization to enable a flexible trade-off between radar and communications
performance and introduce a low-complexity algorithm. The computational costs
of the above three designs are shown to be similar to the conventional
zero-forcing (ZF) precoding. Moreover, to address the more practical constant
modulus waveform design problem, we propose a branch-and-bound algorithm that
obtains a globally optimal solution and derive its worst-case complexity as a
function of the maximum iteration number. Finally, we assess the effectiveness
of the proposed waveform design approaches by numerical results.Comment: 13 pages, 10 figures. This work has been submitted to the IEEE for
possible publication. Copyright may be transferred without notice, after
which this version may no longer be accessibl
Randomized Local Model Order Reduction
In this paper we propose local approximation spaces for localized model order
reduction procedures such as domain decomposition and multiscale methods. Those
spaces are constructed from local solutions of the partial differential
equation (PDE) with random boundary conditions, yield an approximation that
converges provably at a nearly optimal rate, and can be generated at close to
optimal computational complexity. In many localized model order reduction
approaches like the generalized finite element method, static condensation
procedures, and the multiscale finite element method local approximation spaces
can be constructed by approximating the range of a suitably defined transfer
operator that acts on the space of local solutions of the PDE. Optimal local
approximation spaces that yield in general an exponentially convergent
approximation are given by the left singular vectors of this transfer operator
[I. Babu\v{s}ka and R. Lipton 2011, K. Smetana and A. T. Patera 2016]. However,
the direct calculation of these singular vectors is computationally very
expensive. In this paper, we propose an adaptive randomized algorithm based on
methods from randomized linear algebra [N. Halko et al. 2011], which constructs
a local reduced space approximating the range of the transfer operator and thus
the optimal local approximation spaces. The adaptive algorithm relies on a
probabilistic a posteriori error estimator for which we prove that it is both
efficient and reliable with high probability. Several numerical experiments
confirm the theoretical findings.Comment: 31 pages, 14 figures, 1 table, 1 algorith
Nonlinear functional regression by functional deep neural network with kernel embedding
With the rapid development of deep learning in various fields of science and
technology, such as speech recognition, image classification, and natural
language processing, recently it is also widely applied in the functional data
analysis (FDA) with some empirical success. However, due to the infinite
dimensional input, we need a powerful dimension reduction method for functional
learning tasks, especially for the nonlinear functional regression. In this
paper, based on the idea of smooth kernel integral transformation, we propose a
functional deep neural network with an efficient and fully data-dependent
dimension reduction method. The architecture of our functional net consists of
a kernel embedding step: an integral transformation with a data-dependent
smooth kernel; a projection step: a dimension reduction by projection with
eigenfunction basis based on the embedding kernel; and finally an expressive
deep ReLU neural network for the prediction. The utilization of smooth kernel
embedding enables our functional net to be discretization invariant, efficient,
and robust to noisy observations, capable of utilizing information in both
input functions and responses data, and have a low requirement on the number of
discrete points for an unimpaired generalization performance. We conduct
theoretical analysis including approximation error and generalization error
analysis, and numerical simulations to verify these advantages of our
functional net
Robust equalization of multichannel acoustic systems
In most real-world acoustical scenarios, speech signals captured by distant microphones from a source are reverberated due to multipath propagation, and the reverberation may impair speech intelligibility. Speech dereverberation can be achieved
by equalizing the channels from the source to microphones. Equalization systems can
be computed using estimates of multichannel acoustic impulse responses. However,
the estimates obtained from system identification always include errors; the fact that
an equalization system is able to equalize the estimated multichannel acoustic system does not mean that it is able to equalize the true system. The objective of this
thesis is to propose and investigate robust equalization methods for multichannel
acoustic systems in the presence of system identification errors.
Equalization systems can be computed using the multiple-input/output inverse theorem or multichannel least-squares method. However, equalization systems
obtained from these methods are very sensitive to system identification errors. A
study of the multichannel least-squares method with respect to two classes of characteristic channel zeros is conducted. Accordingly, a relaxed multichannel least-
squares method is proposed. Channel shortening in connection with the multiple-
input/output inverse theorem and the relaxed multichannel least-squares method is
discussed.
Two algorithms taking into account the system identification errors are developed. Firstly, an optimally-stopped weighted conjugate gradient algorithm is
proposed. A conjugate gradient iterative method is employed to compute the equalization system. The iteration process is stopped optimally with respect to system identification errors. Secondly, a system-identification-error-robust equalization
method exploring the use of error models is presented, which incorporates system
identification error models in the weighted multichannel least-squares formulation
- …