926 research outputs found
On linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems
In this paper we propose a distributed dual gradient algorithm for minimizing
linearly constrained separable convex problems and analyze its rate of
convergence. In particular, we prove that under the assumption of strong
convexity and Lipshitz continuity of the gradient of the primal objective
function we have a global error bound type property for the dual problem. Using
this error bound property we devise a fully distributed dual gradient scheme,
i.e. a gradient scheme based on a weighted step size, for which we derive
global linear rate of convergence for both dual and primal suboptimality and
for primal feasibility violation. Many real applications, e.g. distributed
model predictive control, network utility maximization or optimal power flow,
can be posed as linearly constrained separable convex problems for which dual
gradient type methods from literature have sublinear convergence rate. In the
present paper we prove for the first time that in fact we can achieve linear
convergence rate for such algorithms when they are used for solving these
applications. Numerical simulations are also provided to confirm our theory.Comment: 14 pages, 4 figures, submitted to Automatica Journal, February 2014.
arXiv admin note: substantial text overlap with arXiv:1401.4398. We revised
the paper, adding more simulations and checking for typo
An Accelerated Proximal Coordinate Gradient Method and its Application to Regularized Empirical Risk Minimization
We consider the problem of minimizing the sum of two convex functions: one is
smooth and given by a gradient oracle, and the other is separable over blocks
of coordinates and has a simple known structure over each block. We develop an
accelerated randomized proximal coordinate gradient (APCG) method for
minimizing such convex composite functions. For strongly convex functions, our
method achieves faster linear convergence rates than existing randomized
proximal coordinate gradient methods. Without strong convexity, our method
enjoys accelerated sublinear convergence rates. We show how to apply the APCG
method to solve the regularized empirical risk minimization (ERM) problem, and
devise efficient implementations that avoid full-dimensional vector operations.
For ill-conditioned ERM problems, our method obtains improved convergence rates
than the state-of-the-art stochastic dual coordinate ascent (SDCA) method
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
We introduce a framework for quasi-Newton forward--backward splitting
algorithms (proximal quasi-Newton methods) with a metric induced by diagonal
rank- symmetric positive definite matrices. This special type of
metric allows for a highly efficient evaluation of the proximal mapping. The
key to this efficiency is a general proximal calculus in the new metric. By
using duality, formulas are derived that relate the proximal mapping in a
rank- modified metric to the original metric. We also describe efficient
implementations of the proximity calculation for a large class of functions;
the implementations exploit the piece-wise linear nature of the dual problem.
Then, we apply these results to acceleration of composite convex minimization
problems, which leads to elegant quasi-Newton methods for which we prove
convergence. The algorithm is tested on several numerical examples and compared
to a comprehensive list of alternatives in the literature. Our quasi-Newton
splitting algorithm with the prescribed metric compares favorably against
state-of-the-art. The algorithm has extensive applications including signal
processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115
A Parametric Non-Convex Decomposition Algorithm for Real-Time and Distributed NMPC
A novel decomposition scheme to solve parametric non-convex programs as they
arise in Nonlinear Model Predictive Control (NMPC) is presented. It consists of
a fixed number of alternating proximal gradient steps and a dual update per
time step. Hence, the proposed approach is attractive in a real-time
distributed context. Assuming that the Nonlinear Program (NLP) is
semi-algebraic and that its critical points are strongly regular, contraction
of the sequence of primal-dual iterates is proven, implying stability of the
sub-optimality error, under some mild assumptions. Moreover, it is shown that
the performance of the optimality-tracking scheme can be enhanced via a
continuation technique. The efficacy of the proposed decomposition method is
demonstrated by solving a centralised NMPC problem to control a DC motor and a
distributed NMPC program for collaborative tracking of unicycles, both within a
real-time framework. Furthermore, an analysis of the sub-optimality error as a
function of the sampling period is proposed given a fixed computational power.Comment: 16 pages, 9 figure
- …