167 research outputs found
Proximal point algorithms on Hadamard manifolds: linear convergence and finite termination
In the present paper, we consider inexact proximal point algorithms for finding singular points of multivalued vector fields on Hadamard manifolds. The rate of convergence is shown to be linear under the mild assumption of metric subregularity. Furthermore, if the sequence of parameters associated with the iterative scheme converges to 0, then the convergence rate is
superlinear. At the same time, the finite termination of the inexact proximal point algorithm is also provided under a weak sharp minima-like condition. Applications to optimization problems are provided. Some of our results are new even in Euclidean spaces, while others improve and/or extend some known results in Euclidean spaces. As a matter of fact, in the case of exact proximal point algorithm, our results improve the corresponding results in [G. C. Bento and J. X. Cruz Neto, Optim., 63 (2014), pp. 1281–1288]. Finally, several examples are provided to illustrate that our results are applicable while the corresponding results in the Hilbert space setting are not.National Natural Science Foundation of ChinaZhejiang Provincial Natural Science Foundation of ChinaDirección General de Enseñanza SuperiorJunta de AndalucÃaNational Science Council of Taiwa
Splitting methods with variable metric for KL functions
We study the convergence of general abstract descent methods applied to a
lower semicontinuous nonconvex function f that satisfies the
Kurdyka-Lojasiewicz inequality in a Hilbert space. We prove that any precompact
sequence converges to a critical point of f and obtain new convergence rates
both for the values and the iterates. The analysis covers alternating versions
of the forward-backward method with variable metric and relative errors. As an
example, a nonsmooth and nonconvex version of the Levenberg-Marquardt algorithm
is detailled
Activity Identification and Local Linear Convergence of Forward--Backward-type methods
In this paper, we consider a class of Forward--Backward (FB) splitting
methods that includes several variants (e.g. inertial schemes, FISTA) for
minimizing the sum of two proper convex and lower semi-continuous functions,
one of which has a Lipschitz continuous gradient, and the other is partly
smooth relatively to a smooth active manifold . We propose a
unified framework, under which we show that, this class of FB-type algorithms
(i) correctly identifies the active manifolds in a finite number of iterations
(finite activity identification), and (ii) then enters a local linear
convergence regime, which we characterize precisely in terms of the structure
of the underlying active manifolds. For simpler problems involving polyhedral
functions, we show finite termination. We also establish and explain why FISTA
(with convergent sequences) locally oscillates and can be slower than FB. These
results may have numerous applications including in signal/image processing,
sparse recovery and machine learning. Indeed, the obtained results explain the
typical behaviour that has been observed numerically for many problems in these
fields such as the Lasso, the group Lasso, the fused Lasso and the nuclear norm
regularization to name only a few.Comment: Full length version of the previous short on
An asymptotically superlinearly convergent semismooth Newton augmented Lagrangian method for Linear Programming
Powerful interior-point methods (IPM) based commercial solvers, such as
Gurobi and Mosek, have been hugely successful in solving large-scale linear
programming (LP) problems. The high efficiency of these solvers depends
critically on the sparsity of the problem data and advanced matrix
factorization techniques. For a large scale LP problem with data matrix
that is dense (possibly structured) or whose corresponding normal matrix
has a dense Cholesky factor (even with re-ordering), these solvers may require
excessive computational cost and/or extremely heavy memory usage in each
interior-point iteration. Unfortunately, the natural remedy, i.e., the use of
iterative methods based IPM solvers, although can avoid the explicit
computation of the coefficient matrix and its factorization, is not practically
viable due to the inherent extreme ill-conditioning of the large scale normal
equation arising in each interior-point iteration. To provide a better
alternative choice for solving large scale LPs with dense data or requiring
expensive factorization of its normal equation, we propose a semismooth Newton
based inexact proximal augmented Lagrangian ({\sc Snipal}) method. Different
from classical IPMs, in each iteration of {\sc Snipal}, iterative methods can
efficiently be used to solve simpler yet better conditioned semismooth Newton
linear systems. Moreover, {\sc Snipal} not only enjoys a fast asymptotic
superlinear convergence but is also proven to enjoy a finite termination
property. Numerical comparisons with Gurobi have demonstrated encouraging
potential of {\sc Snipal} for handling large-scale LP problems where the
constraint matrix has a dense representation or has a dense
factorization even with an appropriate re-ordering.Comment: Due to the limitation "The abstract field cannot be longer than 1,920
characters", the abstract appearing here is slightly shorter than that in the
PDF fil
On starting and stopping criteria for nested primal-dual iterations
The importance of an adequate inner loop starting point (as opposed to a
sufficient inner loop stopping rule) is discussed in the context of a numerical
optimization algorithm consisting of nested primal-dual proximal-gradient
iterations. While the number of inner iterations is fixed in advance,
convergence of the whole algorithm is still guaranteed by virtue of a
warm-start strategy for the inner loop, showing that inner loop "starting
rules" can be just as effective as "stopping rules" for guaranteeing
convergence. The algorithm itself is applicable to the numerical solution of
convex optimization problems defined by the sum of a differentiable term and
two possibly non-differentiable terms. One of the latter terms should take the
form of the composition of a linear map and a proximable function, while the
differentiable term needs an accessible gradient. The algorithm reduces to the
classical proximal gradient algorithm in certain special cases and it also
generalizes other existing algorithms. In addition, under some conditions of
strong convexity, we show a linear rate of convergence.Comment: 18 pages, no figure
- …