2,157 research outputs found
Calculus of the exponent of Kurdyka-{\L}ojasiewicz inequality and its applications to linear convergence of first-order methods
In this paper, we study the Kurdyka-{\L}ojasiewicz (KL) exponent, an
important quantity for analyzing the convergence rate of first-order methods.
Specifically, we develop various calculus rules to deduce the KL exponent of
new (possibly nonconvex and nonsmooth) functions formed from functions with
known KL exponents. In addition, we show that the well-studied Luo-Tseng error
bound together with a mild assumption on the separation of stationary values
implies that the KL exponent is . The Luo-Tseng error bound is known
to hold for a large class of concrete structured optimization problems, and
thus we deduce the KL exponent of a large class of functions whose exponents
were previously unknown. Building upon this and the calculus rules, we are then
able to show that for many convex or nonconvex optimization models for
applications such as sparse recovery, their objective function's KL exponent is
. This includes the least squares problem with smoothly clipped
absolute deviation (SCAD) regularization or minimax concave penalty (MCP)
regularization and the logistic regression problem with
regularization. Since many existing local convergence rate analysis for
first-order methods in the nonconvex scenario relies on the KL exponent, our
results enable us to obtain explicit convergence rate for various first-order
methods when they are applied to a large variety of practical optimization
models. Finally, we further illustrate how our results can be applied to
establishing local linear convergence of the proximal gradient algorithm and
the inertial proximal algorithm with constant step-sizes for some specific
models that arise in sparse recovery.Comment: The paper is accepted for publication in Foundations of Computational
Mathematics: https://link.springer.com/article/10.1007/s10208-017-9366-8. In
this update, we fill in more details to the proof of Theorem 4.1 concerning
the nonemptiness of the projection onto the set of stationary point
- …