5,329 research outputs found
D-ADMM: A Communication-Efficient Distributed Algorithm For Separable Optimization
We propose a distributed algorithm, named Distributed Alternating Direction
Method of Multipliers (D-ADMM), for solving separable optimization problems in
networks of interconnected nodes or agents. In a separable optimization problem
there is a private cost function and a private constraint set at each node. The
goal is to minimize the sum of all the cost functions, constraining the
solution to be in the intersection of all the constraint sets. D-ADMM is proven
to converge when the network is bipartite or when all the functions are
strongly convex, although in practice, convergence is observed even when these
conditions are not met. We use D-ADMM to solve the following problems from
signal processing and control: average consensus, compressed sensing, and
support vector machines. Our simulations show that D-ADMM requires less
communications than state-of-the-art algorithms to achieve a given accuracy
level. Algorithms with low communication requirements are important, for
example, in sensor networks, where sensors are typically battery-operated and
communicating is the most energy consuming operation.Comment: To appear in IEEE Transactions on Signal Processin
Distributed Model Predictive Consensus via the Alternating Direction Method of Multipliers
We propose a distributed optimization method for solving a distributed model
predictive consensus problem. The goal is to design a distributed controller
for a network of dynamical systems to optimize a coupled objective function
while respecting state and input constraints. The distributed optimization
method is an augmented Lagrangian method called the Alternating Direction
Method of Multipliers (ADMM), which was introduced in the 1970s but has seen a
recent resurgence in the context of dramatic increases in computing power and
the development of widely available distributed computing platforms. The method
is applied to position and velocity consensus in a network of double
integrators. We find that a few tens of ADMM iterations yield closed-loop
performance near what is achieved by solving the optimization problem
centrally. Furthermore, the use of recent code generation techniques for
solving local subproblems yields fast overall computation times.Comment: 7 pages, 5 figures, 50th Allerton Conference on Communication,
Control, and Computing, Monticello, IL, USA, 201
On the Convergence of Alternating Direction Lagrangian Methods for Nonconvex Structured Optimization Problems
Nonconvex and structured optimization problems arise in many engineering
applications that demand scalable and distributed solution methods. The study
of the convergence properties of these methods is in general difficult due to
the nonconvexity of the problem. In this paper, two distributed solution
methods that combine the fast convergence properties of augmented
Lagrangian-based methods with the separability properties of alternating
optimization are investigated. The first method is adapted from the classic
quadratic penalty function method and is called the Alternating Direction
Penalty Method (ADPM). Unlike the original quadratic penalty function method,
in which single-step optimizations are adopted, ADPM uses an alternating
optimization, which in turn makes it scalable. The second method is the
well-known Alternating Direction Method of Multipliers (ADMM). It is shown that
ADPM for nonconvex problems asymptotically converges to a primal feasible point
under mild conditions and an additional condition ensuring that it
asymptotically reaches the standard first order necessary conditions for local
optimality are introduced. In the case of the ADMM, novel sufficient conditions
under which the algorithm asymptotically reaches the standard first order
necessary conditions are established. Based on this, complete convergence of
ADMM for a class of low dimensional problems are characterized. Finally, the
results are illustrated by applying ADPM and ADMM to a nonconvex localization
problem in wireless sensor networks.Comment: 13 pages, 6 figure
On linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems
In this paper we propose a distributed dual gradient algorithm for minimizing
linearly constrained separable convex problems and analyze its rate of
convergence. In particular, we prove that under the assumption of strong
convexity and Lipshitz continuity of the gradient of the primal objective
function we have a global error bound type property for the dual problem. Using
this error bound property we devise a fully distributed dual gradient scheme,
i.e. a gradient scheme based on a weighted step size, for which we derive
global linear rate of convergence for both dual and primal suboptimality and
for primal feasibility violation. Many real applications, e.g. distributed
model predictive control, network utility maximization or optimal power flow,
can be posed as linearly constrained separable convex problems for which dual
gradient type methods from literature have sublinear convergence rate. In the
present paper we prove for the first time that in fact we can achieve linear
convergence rate for such algorithms when they are used for solving these
applications. Numerical simulations are also provided to confirm our theory.Comment: 14 pages, 4 figures, submitted to Automatica Journal, February 2014.
arXiv admin note: substantial text overlap with arXiv:1401.4398. We revised
the paper, adding more simulations and checking for typo
- …