3,086 research outputs found
Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods
Our goal is to improve variance reducing stochastic methods through better
control variates. We first propose a modification of SVRG which uses the
Hessian to track gradients over time, rather than to recondition, increasing
the correlation of the control variates and leading to faster theoretical
convergence close to the optimum. We then propose accurate and computationally
efficient approximations to the Hessian, both using a diagonal and a low-rank
matrix. Finally, we demonstrate the effectiveness of our method on a wide range
of problems.Comment: 17 pages, 2 figures, 1 tabl
Detecting Generalized Synchronization Between Chaotic Signals: A Kernel-based Approach
A unified framework for analyzing generalized synchronization in coupled
chaotic systems from data is proposed. The key of the proposed approach is the
use of the kernel methods recently developed in the field of machine learning.
Several successful applications are presented, which show the capability of the
kernel-based approach for detecting generalized synchronization. It is also
shown that the dynamical change of the coupling coefficient between two chaotic
systems can be captured by the proposed approach.Comment: 20 pages, 15 figures. massively revised as a full paper; issues on
the choice of parameters by cross validation, tests by surrogated data, etc.
are added as well as additional examples and figure
Stochastic methods for solving high-dimensional partial differential equations
We propose algorithms for solving high-dimensional Partial Differential
Equations (PDEs) that combine a probabilistic interpretation of PDEs, through
Feynman-Kac representation, with sparse interpolation. Monte-Carlo methods and
time-integration schemes are used to estimate pointwise evaluations of the
solution of a PDE. We use a sequential control variates algorithm, where
control variates are constructed based on successive approximations of the
solution of the PDE. Two different algorithms are proposed, combining in
different ways the sequential control variates algorithm and adaptive sparse
interpolation. Numerical examples will illustrate the behavior of these
algorithms
A Comparison of Relaxations of Multiset Cannonical Correlation Analysis and Applications
Canonical correlation analysis is a statistical technique that is used to
find relations between two sets of variables. An important extension in pattern
analysis is to consider more than two sets of variables. This problem can be
expressed as a quadratically constrained quadratic program (QCQP), commonly
referred to Multi-set Canonical Correlation Analysis (MCCA). This is a
non-convex problem and so greedy algorithms converge to local optima without
any guarantees on global optimality. In this paper, we show that despite being
highly structured, finding the optimal solution is NP-Hard. This motivates our
relaxation of the QCQP to a semidefinite program (SDP). The SDP is convex, can
be solved reasonably efficiently and comes with both absolute and
output-sensitive approximation quality. In addition to theoretical guarantees,
we do an extensive comparison of the QCQP method and the SDP relaxation on a
variety of synthetic and real world data. Finally, we present two useful
extensions: we incorporate kernel methods and computing multiple sets of
canonical vectors
- …