11,154 research outputs found
Distributed Big-Data Optimization via Block Communications
We study distributed multi-agent large-scale optimization problems, wherein
the cost function is composed of a smooth possibly nonconvex sum-utility plus a
DC (Difference-of-Convex) regularizer. We consider the scenario where the
dimension of the optimization variables is so large that optimizing and/or
transmitting the entire set of variables could cause unaffordable computation
and communication overhead. To address this issue, we propose the first
distributed algorithm whereby agents optimize and communicate only a portion of
their local variables. The scheme hinges on successive convex approximation
(SCA) to handle the nonconvexity of the objective function, coupled with a
novel block-signal tracking scheme, aiming at locally estimating the average of
the agents' gradients. Asymptotic convergence to stationary solutions of the
nonconvex problem is established. Numerical results on a sparse regression
problem show the effectiveness of the proposed algorithm and the impact of the
block size on its practical convergence speed and communication cost
A Distributed Asynchronous Method of Multipliers for Constrained Nonconvex Optimization
This paper presents a fully asynchronous and distributed approach for
tackling optimization problems in which both the objective function and the
constraints may be nonconvex. In the considered network setting each node is
active upon triggering of a local timer and has access only to a portion of the
objective function and to a subset of the constraints. In the proposed
technique, based on the method of multipliers, each node performs, when it
wakes up, either a descent step on a local augmented Lagrangian or an ascent
step on the local multiplier vector. Nodes realize when to switch from the
descent step to the ascent one through an asynchronous distributed logic-AND,
which detects when all the nodes have reached a predefined tolerance in the
minimization of the augmented Lagrangian. It is shown that the resulting
distributed algorithm is equivalent to a block coordinate descent for the
minimization of the global augmented Lagrangian. This allows one to extend the
properties of the centralized method of multipliers to the considered
distributed framework. Two application examples are presented to validate the
proposed approach: a distributed source localization problem and the parameter
estimation of a neural network.Comment: arXiv admin note: substantial text overlap with arXiv:1803.0648
Gradient-Free Distributed Optimization with Exact Convergence
In this paper, a gradient-free distributed algorithm is introduced to solve a
set constrained optimization problem under a directed communication network.
Specifically, at each time-step, the agents locally compute a so-called
pseudo-gradient to guide the updates of the decision variables, which can be
applied in the fields where the gradient information is unknown, not available
or non-existent. A surplus-based method is adopted to remove the doubly
stochastic requirement on the weighting matrix, which enables the
implementation of the algorithm in graphs having no associated doubly
stochastic weighting matrix. For the convergence results, the proposed
algorithm is able to obtain the exact convergence to the optimal value with any
positive, non-summable and non-increasing step-sizes. Furthermore, when the
step-size is also square-summable, the proposed algorithm is guaranteed to
achieve the exact convergence to an optimal solution. In addition to the
standard convergence analysis, the convergence rate of the proposed algorithm
with respect to different cases of step-sizes is investigated. Finally, the
effectiveness of the proposed algorithm is verified through numerical
simulations
Gradient-Free Nash Equilibrium Seeking in N-Cluster Games with Uncoordinated Constant Step-Sizes
In this paper, we consider a problem of simultaneous global cost minimization
and Nash equilibrium seeking, which commonly exists in -cluster
non-cooperative games. Specifically, the agents in the same cluster collaborate
to minimize a global cost function, being a summation of their individual cost
functions, and jointly play a non-cooperative game with other clusters as
players. For the problem settings, we suppose that the explicit analytical
expressions of the agents' local cost functions are unknown, but the function
values can be measured. We propose a gradient-free Nash equilibrium seeking
algorithm by a synthesis of Gaussian smoothing techniques and gradient
tracking. Furthermore, instead of using the uniform coordinated step-size, we
allow the agents across different clusters to choose different constant
step-sizes. When the largest step-size is sufficiently small, we prove a linear
convergence of the agents' actions to a neighborhood of the unique Nash
equilibrium under a strongly monotone game mapping condition, with the error
gap being propotional to the largest step-size and the smoothing parameter. The
performance of the proposed algorithm is validated by numerical simulations
- …