Distributed Big-Data Optimization via Block-Iterative Convexification and Averaging

In this paper, we study distributed big-data nonconvex optimization in multi-agent networks. We consider the (constrained) minimization of the sum of a smooth (possibly) nonconvex function, i.e., the agents' sum-utility, plus a convex (possibly) nonsmooth regularizer. Our interest is in big-data problems wherein there is a large number of variables to optimize. If treated by means of standard distributed optimization algorithms, these large-scale problems may be intractable, due to the prohibitive local computation and communication burden at each node. We propose a novel distributed solution method whereby at each iteration agents optimize and then communicate (in an uncoordinated fashion) only a subset of their decision variables. To deal with non-convexity of the cost function, the novel scheme hinges on Successive Convex Approximation (SCA) techniques coupled with i) a tracking mechanism instrumental to locally estimate gradient averages; and ii) a novel block-wise consensus-based protocol to perform local block-averaging operations and gradient tacking. Asymptotic convergence to stationary solutions of the nonconvex problem is established. Finally, numerical results show the effectiveness of the proposed algorithm and highlight how the block dimension impacts on the communication overhead and practical convergence speed

Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

In principle, reinforcement learning and policy search methods can enable robots to learn highly complex and general skills that may allow them to function amid the complexity and diversity of the real world. However, training a policy that generalizes well across a wide range of real-world conditions requires far greater quantity and diversity of experience than is practical to collect with a single robot. Fortunately, it is possible for multiple robots to share their experience with one another, and thereby, learn a policy collectively. In this work, we explore distributed and asynchronous policy learning as a means to achieve generalization and improved training times on challenging, real-world manipulation tasks. We propose a distributed and asynchronous version of Guided Policy Search and use it to demonstrate collective policy learning on a vision-based door opening task using four robots. We show that it achieves better generalization, utilization, and training times than the single robot alternative.Comment: Submitted to the IEEE International Conference on Robotics and Automation 201

Distributed Stochastic Optimization under Imperfect Information

We consider a stochastic convex optimization problem that requires minimizing a sum of misspecified agentspecific expectation-valued convex functions over the intersection of a collection of agent-specific convex sets. This misspecification is manifested in a parametric sense and may be resolved through solving a distinct stochastic convex learning problem. Our interest lies in the development of distributed algorithms in which every agent makes decisions based on the knowledge of its objective and feasibility set while learning the decisions of other agents by communicating with its local neighbors over a time-varying connectivity graph. While a significant body of research currently exists in the context of such problems, we believe that the misspecified generalization of this problem is both important and has seen little study, if at all. Accordingly, our focus lies on the simultaneous resolution of both problems through a joint set of schemes that combine three distinct steps: (i) An alignment step in which every agent updates its current belief by averaging over the beliefs of its neighbors; (ii) A projected (stochastic) gradient step in which every agent further updates this averaged estimate; and (iii) A learning step in which agents update their belief of the misspecified parameter by utilizing a stochastic gradient step. Under an assumption of mere convexity on agent objectives and strong convexity of the learning problems, we show that the sequences generated by this collection of update rules converge almost surely to the solution of the correctly specified stochastic convex optimization problem and the stochastic learning problem, respectively

A randomized primal distributed algorithm for partitioned and big-data non-convex optimization

In this paper we consider a distributed optimization scenario in which the aggregate objective function to minimize is partitioned, big-data and possibly non-convex. Specifically, we focus on a set-up in which the dimension of the decision variable depends on the network size as well as the number of local functions, but each local function handled by a node depends only on a (small) portion of the entire optimization variable. This problem set-up has been shown to appear in many interesting network application scenarios. As main paper contribution, we develop a simple, primal distributed algorithm to solve the optimization problem, based on a randomized descent approach, which works under asynchronous gossip communication. We prove that the proposed asynchronous algorithm is a proper, ad-hoc version of a coordinate descent method and thus converges to a stationary point. To show the effectiveness of the proposed algorithm, we also present numerical simulations on a non-convex quadratic program, which confirm the theoretical results