29 research outputs found
A Bregman forward-backward linesearch algorithm for nonconvex composite optimization: superlinear convergence to nonisolated local minima
We introduce Bella, a locally superlinearly convergent Bregman forward
backward splitting method for minimizing the sum of two nonconvex functions,
one of which satisfying a relative smoothness condition and the other one
possibly nonsmooth. A key tool of our methodology is the Bregman
forward-backward envelope (BFBE), an exact and continuous penalty function with
favorable first- and second-order properties, and enjoying a nonlinear error
bound when the objective function satisfies a Lojasiewicz-type property. The
proposed algorithm is of linesearch type over the BFBE along candidate update
directions, and converges subsequentially to stationary points, globally under
a KL condition, and owing to the given nonlinear error bound can attain
superlinear convergence rates even when the limit point is a nonisolated
minimum, provided the directions are suitably selected
Optimization and Applications
[no abstract available
Local Convergence of Newton-type Methods for Nonsmooth Constrained Equations and Applications
In this thesis we consider constrained systems of equations. The focus is on local Newton-type methods for the solution of constrained systems which converge locally quadratically under mild assumptions implying neither local uniqueness of solutions nor differentiability of the equation function at solutions.
The first aim of this thesis is to improve existing local convergence results of the constrained Levenberg-Marquardt method. To this end, we describe a general Newton-type algorithm. Then we prove local quadratic convergence of this general algorithm under the same four assumptions which were recently used for the local convergence analysis of the LP-Newton method. Afterwards, we show that, besides the LP-Newton method, the constrained Levenberg-Marquardt method can be regarded as a special realization of the general Newton-type algorithm and therefore enjoys the same local convergence properties. Thus, local quadratic convergence of a nonsmooth constrained Levenberg-Marquardt method is proved without requiring conditions implying the local uniqueness of solutions.
As already mentioned, we use four assumptions for the local convergence analysis of the general Newton-type algorithm. The second aim of this thesis is a detailed discussion of these convergence assumptions for the case that the equation function of the constrained system is piecewise continuously differentiable. Some of the convergence assumptions seem quite technical and difficult to check. Therefore, we look for sufficient conditions which are still mild but which seem to be more familiar. We will particularly prove that the whole set of the convergence assumptions holds if some set of local error bound conditions is satisfied and in addition the feasible set of the constrained system excludes those zeros of the selection functions which are not zeros of the equation function itself, at least in a sufficiently small neighborhood of some fixed solution.
We apply our results to constrained systems arising from complementarity systems, i.e., systems of equations and inequalities which contain complementarity constraints. Our new conditions are discussed for a suitable reformulation of the complementarity system as constrained system of equations by means of the minimum function. In particular, it will turn out that the whole set of the convergence assumptions is actually implied by some set of local error bound conditions. In addition, we provide a new constant rank condition implying the whole set of the convergence assumptions.
Particularly, we provide adapted formulations of our new conditions for special classes of complementarity systems. We consider Karush-Kuhn-Tucker (KKT) systems arising from optimization problems, variational inequalities, or generalized Nash equilibrium problems (GNEPs) and Fritz-John (FJ) systems arising from GNEPs. Thus, we obtain for each problem class conditions which guarantee local quadratic convergence of the general Newton-type algorithm and its special realizations to a solution of the particular problem. Moreover, we prove for FJ systems of GNEPs that generically some full row rank condition is satisfied at any solution of the FJ system of a GNEP. The latter condition implies the whole set of the convergence assumptions if the functions which characterize the GNEP are sufficiently smooth.
Finally, we describe an idea for a possible globalization of our Newton-type methods, at least for the case that the constrained system arises from a certain smooth reformulation of the KKT system of a GNEP. More precisely, a hybrid method is presented whose local part is the LP-Newton method. The hybrid method turns out to be, under appropriate conditions, both globally and locally quadratically convergent
Levenberg-Marquardt Algorithms for Nonlinear Equations, Multi-objective Optimization, and Complementarity Problems
The Levenberg-Marquardt algorithm is a classical method for solving
nonlinear systems of equations that can come from various applications
in engineering and economics.
Recently, Levenberg-Marquardt methods turned out to be a valuable
principle for obtaining fast convergence to a solution of the nonlinear
system if the classical nonsingularity assumption is replaced by a
weaker error bound condition. In this way also problems with nonisolated
solutions can be treated successfully. Such problems increasingly
arise in engineering applications and in mathematical programming.
In this thesis we use Levenberg-Marquardt algorithms to deal with
nonlinear equations, multi-objective optimization and complementarity
problems. We develop new algorithms for solving these problems
and investigate their convergence properties.
For sufficiently smooth nonlinear equations we provide convergence results
for inexact Levenberg-Marquardt type algorithms. In particular,
a sharp bound on the maximal level of inexactness that is sufficient for
a quadratic (or a superlinear) rate of convergence is derived. Moreover,
the theory developed is used to show quadratic convergence of
a robust projected Levenberg-Marquardt algorithm.
The use of Levenberg-Marquardt type algorithms for unconstrained
multi-objective optimization problems is investigated in detail. In particular,
two globally and locally quadratically convergent algorithms
for these problems are developed. Moreover, assumptions under which
the error bound condition for a Pareto-critical system is fulfilled are
derived.
We also treat nonsmooth equations arising from reformulating complementarity
problems by means of NCP functions. For these reformulations,
we show that existing smoothness conditions are not satisfied
at degenerate solutions. Moreover, we derive new results for positively
homogeneous functions. The latter results are used to show that appropriate
weaker smoothness conditions (enabling a local Q-quadratic
rate of convergence) hold for certain reformulations.Der Levenberg-Marquardt-Algorithmus ist ein klassisches Verfahren zur Lösung von nichtlinearen Gleichungssystemen, welches in verschiedenen Anwendungen der Ingenieur-und Wirtschaftswissenschaften vorkommen kann. Kürzlich, erwies sich das
Verfahren als ein wertvolles Instrument für die Gewährleistung einer schnelleren Konvergenz für eine Lösung des nichtlinearen Systems, wenn die klassische nichtsinguläre Annahme durch eine schwächere Fehlerschranke der eingebundenen Bedingung ersetzt wird. Auf diese Weise, lassen sich ebenfalls Probleme mit nicht isolierten Lösungen erfolgreich behandeln. Solche Probleme ergeben sich
zunehmend in den praktischen, ingenieurwissenschaftlichen Anwendungen und in der mathematischen Programmierung. In dieser Arbeit verwenden wir Levenberg-Marquardt-
Algorithmus für nichtlinearere Gleichungen, multikriterielle Optimierung - und nichtlineare Komplementaritätsprobleme. Wir entwickeln neue Algorithmen zur Lösung dieser Probleme und untersuchen ihre Konvergenzeigenschaften.
Für ausreichend differenzierbare nichtlineare Gleichungen, analysieren und bieten wir Konvergenzergebnisse für ungenaue Levenberg-Marquardt-Algorithmen Typen. Insbesondere, bieten wir eine strenge Schranke für die maximale Höhe der Ungenauigkeit, die ausreichend ist für eine quadratische (oder eine superlineare) Rate der
Konvergenz. DarĂĽber hinaus, die entwickelte Theorie wird verwendet, um quadratische Konvergenz eines robusten projizierten Levenberg-Marquardt-Algorithmus zu zeigen.
Die Verwendung von Levenberg-Marquardt-Algorithmen Typen für unbeschränkte multikriterielle Optimierungsprobleme im Detail zu untersucht. Insbesondere sind zwei globale und lokale quadratische konvergente Algorithmen für multikriterielle Optimierungsprobleme entwickelt worden. Die Annahmen wurden hergeleitet, unter
welche die Fehlerschranke der eingebundenen Bedingung fĂĽr ein Pareto-kritisches System erfĂĽllt ist.
Wir behandeln auch nicht differenzierbare nichtlineare Gleichungen aus Umformulierung der nichtlinearen Komplementaritätsprobleme durch NCP-Funktionen. Wir zeigen für diese Umformulierungen, dass die bestehenden differenzierbaren Bedingungen nicht
zufrieden mit degenerierten Lösungen sind. Außerdem, leiten wir neue Ergebnisse für positiv homogene NCP-Funktionen. Letztere Ergebnisse werden verwendet um zu zeigen, dass geeignete schwächeren differenzierbare Bedingungen (so dass eine lokale Q-quadratische Konvergenzgeschwindigkeit ermöglichen) für bestimmte
Umformulierungen gelten
On the local convergence of the semismooth Newton method for composite optimization
Existing superlinear convergence rate of the semismooth Newton method relies
on the nonsingularity of the B-Jacobian. This is a strict condition since it
implies that the stationary point to seek is isolated. In this paper, we
consider a large class of nonlinear equations derived from first-order type
methods for solving composite optimization problems. We first present some
equivalent characterizations of the invertibility of the associated B-Jacobian,
providing easy-to-check criteria for the traditional condition. Secondly, we
prove that the strict complementarity and local error bound condition guarantee
a local superlinear convergence rate. The analysis consists of two steps:
showing local smoothness based on partial smoothness or closedness of the set
of nondifferentiable points of the proximal map, and applying the local error
bound condition to the locally smooth nonlinear equations. Concrete examples
satisfying the required assumptions are presented. The main novelty of the
proposed condition is that it also applies to nonisolated stationary points.Comment: 25 page
Newton's Method for Solving Inclusions Using Set-Valued Approximations
International audienceResults on stability of both local and global metric regularity under set-valued perturbations are presented. As an application, we study (super)linear convergence of a Newton- type iterative process for solving generalized equations. We investigate several iterative schemes such as the inexact Newton’s method, the nonsmooth Newton’s method for semismooth functions, the inexact proximal point algorithm, etc. Moreover, we also cover a forward-backward splitting algorithm for finding a zero of the sum of two multivalued (not necessarily monotone) operators. Finally, a globalization of the Newton’s method is discussed