Accelerated linearized alternating direction method of multipliers with Nesterov extrapolation

The alternating direction method of multipliers (ADMMs) has found widespread use in solving separable convex optimization problems. In this paper, by employing Nesterov extrapolation technique, we propose two families of accelerated linearized ADMMsfor addressing two-block linearly constrained separable convex optimization problems where each block of the objective function exhibits a `nonsmooth' plus `smooth' composite structure. Our proposed accelerated linearized ADMMs extend two classical Nesterov acceleration methods designed for unconstrained composite optimization problems to linearly constrained problems. These methods are capable of achieving non-ergodic convergence rates of O(1/k^2) provided that one block of the objective function exhibits strong convexity and the gradients of smooth terms are Lipschitz continuous. We show that the proposed methods can reduce to accelerated linearized augmented Lagrangian methods (ALMs) with non-ergodic O(1/k2 ) convergence rates for solving one-block linearly constrained convex optimization problems. Furthermore, we explore hybrid versions of the proposed linearized accelerated ADMMs, which also demonstrate non-ergodic convergence rates of O(1/k2) under the same assumption. We also investigate inexact versions of the proposed methods. Under mild assumptions regarding error sequences, these methods still maintain O(1/k2) non-ergodic convergence rates. By choosing different extrapolation parameters, we explore the relationship between the proposed methods and existing accelerated methods

Fast convergence of dynamical ADMM via time scaling of damped inertial dynamics

In this paper, we propose in a Hilbertian setting a second-order time-continuous dynamic system with fast convergence guarantees to solve structured convex minimization problems with an affine constraint. The system is associated with the augmented Lagrangian formulation of the minimization problem. The corresponding dynamics brings into play three general time-varying parameters, each with specific properties, and which are respectively associated with viscous damping, extrapolation and temporal scaling. By appropriately adjusting these parameters, we develop a Lyapunov analysis which provides fast convergence properties of the values and of the feasibility gap. These results will naturally pave the way for developing corresponding accelerated ADMM algorithms, obtained by temporal discretization