A Unified Distributed Method for Constrained Networked Optimization via Saddle-Point Dynamics

This paper develops a unified distributed method for solving two classes of constrained networked optimization problems, i.e., optimal consensus problem and resource allocation problem with non-identical set constraints. We first transform these two constrained networked optimization problems into a unified saddle-point problem framework with set constraints. Subsequently, two projection-based primal-dual algorithms via Optimistic Gradient Descent Ascent (OGDA) method and Extra-gradient (EG) method are developed for solving constrained saddle-point problems. It is shown that the developed algorithms achieve exact convergence to a saddle point with an ergodic convergence rate $O(1/k)$ for general convex-concave functions. Based on the proposed primal-dual algorithms via saddle-point dynamics, we develop unified distributed algorithm design and convergence analysis for these two networked optimization problems. Finally, two numerical examples are presented to demonstrate the theoretical results

Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization

In this work, we revisit a classical incremental implementation of the primal-descent dual-ascent gradient method used for the solution of equality constrained optimization problems. We provide a short proof that establishes the linear (exponential) convergence of the algorithm for smooth strongly-convex cost functions and study its relation to the non-incremental implementation. We also study the effect of the augmented Lagrangian penalty term on the performance of distributed optimization algorithms for the minimization of aggregate cost functions over multi-agent networks

Implicit Tracking-based Distributed Constraint-coupled Optimization

A class of distributed optimization problem with a globally coupled equality constraint and local constrained sets is studied in this paper. For its special case where local constrained sets are absent, an augmented primal-dual gradient dynamics is proposed and analyzed, but it cannot be implemented distributedly since the violation of the coupled constraint needs to be used. Benefiting from the brand-new comprehending of a classical distributed unconstrained optimization algorithm, the novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the \underline{i}mplicit tracking-based \underline{d}istribut\underline{e}d \underline{a}ugmented primal-dual gradient dynamics (IDEA). A projected variant of IDEA, i.e., Proj-IDEA, is further designed to deal with the general case where local constrained sets exist. With the aid of the Lyapunov stability theory, the convergences of IDEA and Pro-IDEA over undigraphs and digraphs are analyzed respectively. As far as we know, Proj-IDEA is the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions. Besides, if local cost functions are strongly convex and smooth, IDEA can achieve exponential convergence with a weaker condition about the coupled constraint. Finally, numerical experiments are taken to corroborate our theoretical results.Comment: in IEEE Transactions on Control of Network Systems, 202