Primal superlinear convergence of SQP methods in piecewise linear-quadratic composite optimization

This paper mainly concerns with the primal superlinear convergence of the quasi-Newton sequential quadratic programming (SQP) method for piecewise linear-quadratic composite optimization problems. We show that the latter primal superlinear convergence can be justified under the noncriticality of Lagrange multipliers and a version of the Dennis-More condition. Furthermore, we show that if we replace the noncriticality condition with the second-order sufficient condition, this primal superlinear convergence is equivalent with an appropriate version of the Dennis-More condition. We also recover Bonnans' result in [1] for the primal-dual superlinear of the basic SQP method for this class of composite problems under the second-order sufficient condition and the uniqueness of Lagrange multipliers. To achieve these goals, we first obtain an extension of the reduction lemma for convex Piecewise linear-quadratic functions and then provide a comprehensive analysis of the noncriticality of Lagrange multipliers for composite problems. We also establish certain primal estimates for KKT systems of composite problems, which play a significant role in our local convergence analysis of the quasi-Newton SQP method.Comment: 36 page

Second-order analysis of piecewise linear functions with applications to optimization and stability

This paper is devoted to second-order variational analysis of a rather broad class of extended-real-valued piecewise liner functions and their applications to various issues of optimization and stability. Based on our recent explicit calculations of the second-order subdifferential for such functions, we establish relationships between nondegeneracy and second-order qualification for fully amenable compositions involving piecewise linear functions. We then provide a second-order characterization of full stable local minimizers in composite optimization and constrained minimax problems.Comment: 29 page

Generalized differentiation of piecewise linear functions in second-order variational analysis

The paper is devoted to a comprehensive second-order study of a remarkable class of convex extended-real-valued functions that is highly important in many aspects of nonlinear and variational analysis, specifically those related to optimization and stability. This class consists of lower semicontinuous functions with possibly infinite values on finite-dimensional spaces, which are labeled as piecewise linear ones and can be equivalently described via the convexity of their epigraphs. In this the paper we calculate the second-order subdifferentials (generalized Hessians) of arbitrary convex piecewise linear functions, together with the corresponding geometric objects, entirely in terms of their initial data. The obtained formulas allow us, in particular, to justify a new exact (equality-type) second-order sum rule for such functions in the general nonsmooth setting.Comment: 33 page

Criticality of Lagrange Multipliers in Extended Nonlinear Optimization

The paper is devoted to the study and applications of criticality of Lagrange multipliers in variational systems, which are associated with the class of problems in composite optimization known as extended nonlinear programming (ENLP). The importance of both ENLP and the concept of multiplier criticality in variational systems has been recognized in theoretical and numerical aspects of optimization and variational analysis, while the criticality notion has never been investigated in the ENLP framework. We present here a systematic study of critical and noncritical multipliers in a general variational setting that covers, in particular, KKT systems in ENLP with establishing their verifiable characterizations as well as relationships between noncriticality and other stability notions in variational analysis. Our approach is mainly based on advanced tools of second-order variational analysis and generalized differentiation

Criticality of Lagrange Multipliers in Variational Systems

The paper concerns the study of criticality of Lagrange multipliers in variational systems that has been recognized in both theoretical and numerical aspects of optimization and variational analysis. In contrast to the previous developments dealing with polyhedral KKT systems and the like, we now focus on general nonpolyhedral systems that are associated, in particular, with problems of conic programming. Developing a novel approach, which is mainly based on advanced techniques and tools of second-order variational analysis and generalized differentiation, allows us to overcome principal challenges of nonpolyhedrality and to establish complete characterizations on noncritical multipliers in such settings. The obtained results are illustrated by examples from semidefinite programming

Variational Analysis of Composite Models with Applications to Continuous Optimization

The paper is devoted to a comprehensive study of composite models in variational analysis and optimization the importance of which for numerous theoretical, algorithmic, and applied issues of operations research is difficult to overstate. The underlying theme of our study is a systematical replacement of conventional metric regularity and related requirements by much weaker metric subregulatity ones that lead us to significantly stronger and completely new results of first-order and second-order variational analysis and optimization. In this way we develop extended calculus rules for first-order and second-order generalized differential constructions with paying the main attention in second-order variational theory to the new and rather large class of fully subamenable compositions. Applications to optimization include deriving enhanced no-gap second order optimality conditions in constrained composite models, complete characterizations of the uniqueness of Lagrange multipliers and strong metric subregularity of KKT systems in parametric optimization, etc

Generalized Newton Algorithms for Tilt-Stable Minimizers in Nonsmooth Optimization

This paper aims at developing two versions of the generalized Newton method to compute not merely arbitrary local minimizers of nonsmooth optimization problems but just those, which possess an important stability property known as tilt stability. We start with unconstrained minimization of continuously differentiable cost functions having Lipschitzian gradients and suggest two second-order algorithms of the Newton type: one involving coderivatives of Lipschitzian gradient mappings, and the other based on graphical derivatives of the latter. Then we proceed with the propagation of these algorithms to minimization of extended-real-valued prox-regular functions, while covering in this way problems of constrained optimization, by using Moreau envelops. Employing advanced techniques of second-order variational analysis and characterizations of tilt stability allows us to establish the solvability of subproblems in both algorithms and to prove the Q-superlinear convergence of their iterations

Twice epi-differentiability of extended-real-valued functions with applications in composite optimization

The paper is devoted to the study of the twice epi-differentiablity of extended-real-valued functions, with an emphasis on functions satisfying a certain composite representation. This will be conducted under the parabolic regularity, a second-order regularity condition that was recently utilized in [13] for second-order variational analysis of constraint systems. Besides justifying the twice epi-differentiablity of composite functions, we obtain precise formulas for their second subderivatives under the metric subregularity constraint qualification. The latter allows us to derive second-order optimality conditions for a large class of composite optimization problems

Second-oder analysis in second-oder cone programming

The paper conducts a second-order variational analysis for an important class of nonpolyhedral conic programs generated by the so-called second-order/Lorentz/ice-cream cone $Q$. From one hand, we prove that the indicator function of $Q$ is always twice epi-differentiable and apply this result to characterizing the uniqueness of Lagrange multipliers at stationary points together with an error bound estimate in the general second-order cone setting involving ${\cal C}^2$-smooth data. On the other hand, we precisely calculate the graphical derivative of the normal cone mapping to $Q$ under the weakest metric subregularity constraint qualification and then give an application of the latter result to a complete characterization of isolated calmness for perturbed variational systems associated with second-order cone programs. The obtained results seem to be the first in the literature in these directions for nonpolyhedral problems without imposing any nondegeneracy assumptions

Stability of KKT systems and superlinear convergence of the SQP method under parabolic regularity

This paper pursues a two-fold goal. Firstly, we aim to derive novel second-order characterizations of important robust stability properties of perturbed Karush-Kuhn-Tucker systems for a broadclass of constrained optimization problems generated by parabolically regular sets. Secondly, the obtained characterizations are applied to establish well-posedness and superlinear convergence of the basic sequential quadratic programming method to solve parabolically regular constrained optimization problems