D-ADMM: A Communication-Efficient Distributed Algorithm For Separable Optimization

We propose a distributed algorithm, named Distributed Alternating Direction Method of Multipliers (D-ADMM), for solving separable optimization problems in networks of interconnected nodes or agents. In a separable optimization problem there is a private cost function and a private constraint set at each node. The goal is to minimize the sum of all the cost functions, constraining the solution to be in the intersection of all the constraint sets. D-ADMM is proven to converge when the network is bipartite or when all the functions are strongly convex, although in practice, convergence is observed even when these conditions are not met. We use D-ADMM to solve the following problems from signal processing and control: average consensus, compressed sensing, and support vector machines. Our simulations show that D-ADMM requires less communications than state-of-the-art algorithms to achieve a given accuracy level. Algorithms with low communication requirements are important, for example, in sensor networks, where sensors are typically battery-operated and communicating is the most energy consuming operation.Comment: To appear in IEEE Transactions on Signal Processin

Cooperative Convex Optimization in Networked Systems: Augmented Lagrangian Algorithms with Directed Gossip Communication

We study distributed optimization in networked systems, where nodes cooperate to find the optimal quantity of common interest, x=x^\star. The objective function of the corresponding optimization problem is the sum of private (known only by a node,) convex, nodes' objectives and each node imposes a private convex constraint on the allowed values of x. We solve this problem for generic connected network topologies with asymmetric random link failures with a novel distributed, decentralized algorithm. We refer to this algorithm as AL-G (augmented Lagrangian gossiping,) and to its variants as AL-MG (augmented Lagrangian multi neighbor gossiping) and AL-BG (augmented Lagrangian broadcast gossiping.) The AL-G algorithm is based on the augmented Lagrangian dual function. Dual variables are updated by the standard method of multipliers, at a slow time scale. To update the primal variables, we propose a novel, Gauss-Seidel type, randomized algorithm, at a fast time scale. AL-G uses unidirectional gossip communication, only between immediate neighbors in the network and is resilient to random link failures. For networks with reliable communication (i.e., no failures,) the simplified, AL-BG (augmented Lagrangian broadcast gossiping) algorithm reduces communication, computation and data storage cost. We prove convergence for all proposed algorithms and demonstrate by simulations the effectiveness on two applications: l_1-regularized logistic regression for classification and cooperative spectrum sensing for cognitive radio networks.Comment: 28 pages, journal; revise

Resilient Distributed MPC Algorithm for Microgrid Energy Management under Uncertainties

This paper proposes a resilient distributed energy management algorithm able to cope with different types of faults in a DC microgrid system. A distributed optimization method allows to solve the energy management problem without sharing any private data with the network and reducing the computational cost for each agent, with respect to the centralised case. A distributed MPC scheme based on distributed optimization is used to cope with uncertainty that characterizes the microgrid operation. In order to be resilient to faults that limit the amount of power available to consumers, we propose to adaptively store an amount of power in the storage systems to support the loads. A soft constraint on the minimum energy stored in each battery is introduced for feasibility and to cope with persistent faults. The effectiveness of the method is proved by extensive simulation results considering faults on three types of components: renewable generator, distribution grid and communication network

Projected Push-Pull For Distributed Constrained Optimization Over Time-Varying Directed Graphs (extended version)

We introduce the Projected Push-Pull algorithm that enables multiple agents to solve a distributed constrained optimization problem with private cost functions and global constraints, in a collaborative manner. Our algorithm employs projected gradient descent to deal with constraints and a lazy update rule to control the trade-off between the consensus and optimization steps in the protocol. We prove that our algorithm achieves geometric convergence over time-varying directed graphs while ensuring that the decision variable always stays within the constraint set. We derive explicit bounds for step sizes that guarantee geometric convergence based on the strong-convexity and smoothness of cost functions, and graph properties. Moreover, we provide additional theoretical results on the usefulness of lazy updates, revealing the challenges in the analysis of any gradient tracking method that uses projection operators in a distributed constrained optimization setting. We validate our theoretical results with numerical studies over different graph types, showing that our algorithm achieves geometric convergence empirically.Comment: 16 pages, 2 figure

D-ADMM: A distributed algorithm for compressed sensing and other separable optimization problems

We propose a distributed, decentralized algorithm for solving separable optimization problems over a connected network of compute nodes. In a separable problem, each node has its own private function and its own private constraint set. Private means that no other node has access to it. The goal is to minimize the sum of all nodes ’ private functions, constraining the solution to be in the intersection of all the private sets. Our algorithm is based on the alternating direction method of multipliers (ADMM) and requires a coloring of the network to be available beforehand. We perform numerical experiments of the algorithm, applying it to compressed sensing problems. These show that the proposed algorithm requires in general less iterations, and hence less communication between nodes, than previous algorithms to achieve a given accuracy. Index Terms — Distributed optimization, compressed sensing, basis pursuit, network optimizatio