1 research outputs found
Distributed second order methods with increasing number of working nodes
Recently, an idling mechanism has been introduced in the context of
distributed \emph{first order} methods for minimization of a sum of nodes'
local convex costs over a generic, connected network. With the idling
mechanism, each node , at each iteration , is active -- updates its
solution estimate and exchanges messages with its network neighborhood -- with
probability , and it stays idle with probability , while the
activations are independent both across nodes and across iterations. In this
paper, we demonstrate that the idling mechanism can be successfully
incorporated in \emph{distributed second order methods} also. Specifically, we
apply the idling mechanism to the recently proposed Distributed Quasi Newton
method (DQN). We first show theoretically that, when grows to one across
iterations in a controlled manner, DQN with idling exhibits very similar
theoretical convergence and convergence rates properties as the standard DQN
method, thus achieving the same order of convergence rate (R-linear) as the
standard DQN, but with significantly cheaper updates. Simulation examples
confirm the benefits of incorporating the idling mechanism, demonstrate the
method's flexibility with respect to the choice of the 's, and compare the
proposed idling method with related algorithms from the literature