A randomized primal distributed algorithm for partitioned and big-data non-convex optimization

In this paper we consider a distributed optimization scenario in which the aggregate objective function to minimize is partitioned, big-data and possibly non-convex. Specifically, we focus on a set-up in which the dimension of the decision variable depends on the network size as well as the number of local functions, but each local function handled by a node depends only on a (small) portion of the entire optimization variable. This problem set-up has been shown to appear in many interesting network application scenarios. As main paper contribution, we develop a simple, primal distributed algorithm to solve the optimization problem, based on a randomized descent approach, which works under asynchronous gossip communication. We prove that the proposed asynchronous algorithm is a proper, ad-hoc version of a coordinate descent method and thus converges to a stationary point. To show the effectiveness of the proposed algorithm, we also present numerical simulations on a non-convex quadratic program, which confirm the theoretical results

Asynchronous Distributed Optimization Via Randomized Dual Proximal Gradient

In this paper we consider distributed optimization problems in which the cost function is separable, i.e., a sum of possibly non-smooth functions all sharing a common variable, and can be split into a strongly convex term and a convex one. The second term is typically used to encode constraints or to regularize the solution. We propose a class of distributed optimization algorithms based on proximal gradient methods applied to the dual problem. We show that, by choosing suitable primal variable copies, the dual problem is itself separable when written in terms of conjugate functions, and the dual variables can be stacked into non-overlapping blocks associated to the computing nodes. We first show that a weighted proximal gradient on the dual function leads to a synchronous distributed algorithm with local dual proximal gradient updates at each node. Then, as main paper contribution, we develop asynchronous versions of the algorithm in which the node updates are triggered by local timers without any global iteration counter. The algorithms are shown to be proper randomized block-coordinate proximal gradient updates on the dual function

A randomized primal distributed algorithm for partitioned and big-data non-convex optimization

In this paper we consider a distributed opti- mization scenario in which the aggregate objective function to minimize is partitioned, big-data and possibly non-convex. Specifically, we focus on a set-up in which the dimension of the decision variable depends on the network size as well as the number of local functions, but each local function handled by a node depends only on a (small) portion of the entire optimiza- tion variable. This problem set-up has been shown to appear in many interesting network application scenarios. As main paper contribution, we develop a simple, primal distributed algorithm to solve the optimization problem, based on a randomized descent approach, which works under asynchronous gossip communication. We prove that the proposed asynchronous algorithm is a proper, ad-hoc version of a coordinate descent method and thus converges to a stationary point. To show the effectiveness of the proposed algorithm, we also present numerical simulations on a non-convex quadratic program, which confirm the theoretical results

Fast-Convergent Dynamics for Distributed Resource Allocation Over Sparse Time-Varying Networks

In this paper, distributed dynamics are deployed to solve resource allocation over time-varying multi-agent networks. The state of each agent represents the amount of resources used/produced at that agent while the total amount of resources is fixed. The idea is to optimally allocate the resources among the group of agents by reducing the total cost functions subject to fixed amount of total resources. The information of each agent is restricted to its own state and cost function and those of its immediate neighbors. This is motivated by distributed applications such as in mobile edge-computing, economic dispatch over smart grids, and multi-agent coverage control. The non-Lipschitz dynamics proposed in this work shows fast convergence as compared to the linear and some nonlinear solutions in the literature. Further, the multi-agent network connectivity is more relaxed in this paper. To be more specific, the proposed dynamics even reaches optimal solution over time-varying disconnected undirected networks as far as the union of these networks over some bounded non-overlapping time-intervals includes a spanning-tree. The proposed convergence analysis can be applied for similar 1st-order resource allocation nonlinear dynamics. We provide simulations to verify our results