1 research outputs found
Local and Global Convergence of an Inertial Version of Forward-Backward Splitting
A problem of great interest in optimization is to minimize a sum of two
closed, proper, and convex functions where one is smooth and the other has a
computationally inexpensive proximal operator. In this paper we analyze a
family of Inertial Forward-Backward Splitting (I-FBS) algorithms for solving
this problem. We first apply a global Lyapunov analysis to I-FBS and prove weak
convergence of the iterates to a minimizer in a real Hilbert space. We then
show that the algorithms achieve local linear convergence for "sparse
optimization", which is the important special case where the nonsmooth term is
the -norm. This result holds under either a restricted strong convexity
or a strict complimentary condition and we do not require the objective to be
strictly convex. For certain parameter choices we determine an upper bound on
the number of iterations until the iterates are confined on a manifold
containing the solution set and linear convergence holds.
The local linear convergence result for sparse optimization holds for the
Fast Iterative Shrinkage and Soft Thresholding Algorithm (FISTA) due to Beck
and Teboulle which is a particular parameter choice for I-FBS. In spite of its
optimal global objective function convergence rate, we show that FISTA is not
optimal for sparse optimization with respect to the local convergence rate. We
determine the locally optimal parameter choice for the I-FBS family. Finally we
propose a method which inherits the excellent global rate of FISTA but also has
excellent local rate.Comment: The proofs of Thms. 4.1, 5.1, 5.2, and 5.6 of this manuscript contain
several errors. These errors have been fixed in a revised and rewritten
manuscript entitled "Local and Global Convergence of a General Inertial
Proximal Splitting Scheme" arxiv id. 1602.02726. We recommend reading this
updated manuscript, available at arXiv:1602.0272