9,844 research outputs found
Dynamics and Performance of Susceptibility Propagation on Synthetic Data
We study the performance and convergence properties of the Susceptibility
Propagation (SusP) algorithm for solving the Inverse Ising problem. We first
study how the temperature parameter (T) in a Sherrington-Kirkpatrick model
generating the data influences the performance and convergence of the
algorithm. We find that at the high temperature regime (T>4), the algorithm
performs well and its quality is only limited by the quality of the supplied
data. In the low temperature regime (T<4), we find that the algorithm typically
does not converge, yielding diverging values for the couplings. However, we
show that by stopping the algorithm at the right time before divergence becomes
serious, good reconstruction can be achieved down to T~2. We then show that
dense connectivity, loopiness of the connectivity, and high absolute
magnetization all have deteriorating effects on the performance of the
algorithm. When absolute magnetization is high, we show that other methods can
be work better than SusP. Finally, we show that for neural data with high
absolute magnetization, SusP performs less well than TAP inversion.Comment: 9 pages, 7 figure
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice
We introduce a generic scheme for accelerating gradient-based optimization
methods in the sense of Nesterov. The approach, called Catalyst, builds upon
the inexact accelerated proximal point algorithm for minimizing a convex
objective function, and consists of approximately solving a sequence of
well-chosen auxiliary problems, leading to faster convergence. One of the keys
to achieve acceleration in theory and in practice is to solve these
sub-problems with appropriate accuracy by using the right stopping criterion
and the right warm-start strategy. We give practical guidelines to use Catalyst
and present a comprehensive analysis of its global complexity. We show that
Catalyst applies to a large class of algorithms, including gradient descent,
block coordinate descent, incremental algorithms such as SAG, SAGA, SDCA, SVRG,
MISO/Finito, and their proximal variants. For all of these methods, we
establish faster rates using the Catalyst acceleration, for strongly convex and
non-strongly convex objectives. We conclude with extensive experiments showing
that acceleration is useful in practice, especially for ill-conditioned
problems.Comment: link to publisher website:
http://jmlr.org/papers/volume18/17-748/17-748.pd
- …