347 research outputs found
Stability and Error Analysis for Optimization and Generalized Equations
Stability and error analysis remain challenging for problems that lack
regularity properties near solutions, are subject to large perturbations, and
might be infinite dimensional. We consider nonconvex optimization and
generalized equations defined on metric spaces and develop bounds on solution
errors using the truncated Hausdorff distance applied to graphs and epigraphs
of the underlying set-valued mappings and functions. In the process, we extend
the calculus of such distances to cover compositions and other constructions
that arise in nonconvex problems. The results are applied to constrained
problems with feasible sets that might have empty interiors, solution of KKT
systems, and optimality conditions for difference-of-convex functions and
composite functions
Charactarizations of Linear Suboptimality for Mathematical Programs with Equilibrium Constraints
The paper is devoted to the study of a new notion of linear suboptimality in constrained mathematical programming. This concept is different from conventional notions of solutions to optimization-related problems, while seems to be natural and significant from the viewpoint of modern variational analysis and applications. In contrast to standard notions, it admits complete characterizations via appropriate constructions of generalized differentiation in nonconvex settings. In this paper we mainly focus on various classes of mathematical programs with equilibrium constraints (MPECs), whose principal role has been well recognized in optimization theory and its applications. Based on robust generalized differential calculus, we derive new results giving pointwise necessary and sufficient conditions for linear suboptimality in general MPECs and its important specifications involving variational and quasi variational inequalities, implicit complementarity problems, etc
Bilevel Optimization without Lower-Level Strong Convexity from the Hyper-Objective Perspective
Bilevel optimization reveals the inner structure of otherwise oblique
optimization problems, such as hyperparameter tuning and meta-learning. A
common goal in bilevel optimization is to find stationary points of the
hyper-objective function. Although this hyper-objective approach is widely
used, its theoretical properties have not been thoroughly investigated in cases
where the lower-level functions lack strong convexity. In this work, we take a
step forward and study the hyper-objective approach without the typical
lower-level strong convexity assumption. Our hardness results show that the
hyper-objective of general convex lower-level functions can be intractable
either to evaluate or to optimize. To tackle this challenge, we introduce the
gradient dominant condition, which strictly relaxes the strong convexity
assumption by allowing the lower-level solution set to be non-singleton. Under
the gradient dominant condition, we propose the Inexact Gradient-Free Method
(IGFM), which uses the Switching Gradient Method (SGM) as the zeroth order
oracle, to find an approximate stationary point of the hyper-objective. We also
extend our results to nonsmooth lower-level functions under the weak sharp
minimum condition
Strong Metric (Sub)regularity of KKT Mappings for Piecewise Linear-Quadratic Convex-Composite Optimization
This work concerns the local convergence theory of Newton and quasi-Newton
methods for convex-composite optimization: minimize f(x):=h(c(x)), where h is
an infinite-valued proper convex function and c is C^2-smooth. We focus on the
case where h is infinite-valued piecewise linear-quadratic and convex. Such
problems include nonlinear programming, mini-max optimization, estimation of
nonlinear dynamics with non-Gaussian noise as well as many modern approaches to
large-scale data analysis and machine learning. Our approach embeds the
optimality conditions for convex-composite optimization problems into a
generalized equation. We establish conditions for strong metric subregularity
and strong metric regularity of the corresponding set-valued mappings. This
allows us to extend classical convergence of Newton and quasi-Newton methods to
the broader class of non-finite valued piecewise linear-quadratic
convex-composite optimization problems. In particular we establish local
quadratic convergence of the Newton method under conditions that parallel those
in nonlinear programming when h is non-finite valued piecewise linear
On convergence of the maximum block improvement method
Abstract. The MBI (maximum block improvement) method is a greedy approach to solving optimization problems where the decision variables can be grouped into a finite number of blocks. Assuming that optimizing over one block of variables while fixing all others is relatively easy, the MBI method updates the block of variables corresponding to the maximally improving block at each iteration, which is arguably a most natural and simple process to tackle block-structured problems with great potentials for engineering applications. In this paper we establish global and local linear convergence results for this method. The global convergence is established under the Lojasiewicz inequality assumption, while the local analysis invokes second-order assumptions. We study in particular the tensor optimization model with spherical constraints. Conditions for linear convergence of the famous power method for computing the maximum eigenvalue of a matrix follow in this framework as a special case. The condition is interpreted in various other forms for the rank-one tensor optimization model under spherical constraints. Numerical experiments are shown to support the convergence property of the MBI method
- …