846 research outputs found
Delay-agnostic Asynchronous Distributed Optimization
Existing asynchronous distributed optimization algorithms often use
diminishing step-sizes that cause slow practical convergence, or fixed
step-sizes that depend on an assumed upper bound of delays. Not only is such a
delay bound hard to obtain in advance, but it is also large and therefore
results in unnecessarily slow convergence. This paper develops asynchronous
versions of two distributed algorithms, DGD and DGD-ATC, for solving consensus
optimization problems over undirected networks. In contrast to alternatives,
our algorithms can converge to the fixed-point set of their synchronous
counterparts using step-sizes that are independent of the delays. We establish
convergence guarantees under both partial and total asynchrony. The practical
performance of our algorithms is demonstrated by numerical experiments
Asynchrony and Acceleration in Gossip Algorithms
This paper considers the minimization of a sum of smooth and strongly convex
functions dispatched over the nodes of a communication network. Previous works
on the subject either focus on synchronous algorithms, which can be heavily
slowed down by a few slow nodes (the straggler problem), or consider a model of
asynchronous operation (Boyd et al., 2006) in which adjacent nodes communicate
at the instants of Poisson point processes. We have two main contributions. 1)
We propose CACDM (a Continuously Accelerated Coordinate Dual Method), and for
the Poisson model of asynchronous operation, we prove CACDM to converge to
optimality at an accelerated convergence rate in the sense of Nesterov et
Stich, 2017. In contrast, previously proposed asynchronous algorithms have not
been proven to achieve such accelerated rate. While CACDM is based on discrete
updates, the proof of its convergence crucially depends on a continuous time
analysis. 2) We introduce a new communication scheme based on Loss-Networks,
that is programmable in a fully asynchronous and decentralized way, unlike the
Poisson model of asynchronous operation that does not capture essential aspects
of asynchrony such as non-instantaneous communications and computations. Under
this Loss-Network model of asynchrony, we establish for CDM (a Coordinate Dual
Method) a rate of convergence in terms of the eigengap of the Laplacian of the
graph weighted by local effective delays. We believe this eigengap to be a
fundamental bottleneck for convergence rates of asynchronous optimization.
Finally, we verify empirically that CACDM enjoys an accelerated convergence
rate in the Loss-Network model of asynchrony
- …