3,808 research outputs found
Semi-Global Exponential Stability of Augmented Primal-Dual Gradient Dynamics for Constrained Convex Optimization
Primal-dual gradient dynamics that find saddle points of a Lagrangian have
been widely employed for handling constrained optimization problems. Building
on existing methods, we extend the augmented primal-dual gradient dynamics
(Aug-PDGD) to incorporate general convex and nonlinear inequality constraints,
and we establish its semi-global exponential stability when the objective
function is strongly convex. We also provide an example of a strongly convex
quadratic program of which the Aug-PDGD fails to achieve global exponential
stability. Numerical simulation also suggests that the exponential convergence
rate could depend on the initial distance to the KKT point
An SDP Approach For Solving Quadratic Fractional Programming Problems
This paper considers a fractional programming problem (P) which minimizes a
ratio of quadratic functions subject to a two-sided quadratic constraint. As is
well-known, the fractional objective function can be replaced by a parametric
family of quadratic functions, which makes (P) highly related to, but more
difficult than a single quadratic programming problem subject to a similar
constraint set. The task is to find the optimal parameter and then
look for the optimal solution if is attained. Contrasted with the
classical Dinkelbach method that iterates over the parameter, we propose a
suitable constraint qualification under which a new version of the S-lemma with
an equality can be proved so as to compute directly via an exact
SDP relaxation. When the constraint set of (P) is degenerated to become an
one-sided inequality, the same SDP approach can be applied to solve (P) {\it
without any condition}. We observe that the difference between a two-sided
problem and an one-sided problem lies in the fact that the S-lemma with an
equality does not have a natural Slater point to hold, which makes the former
essentially more difficult than the latter. This work does not, either, assume
the existence of a positive-definite linear combination of the quadratic terms
(also known as the dual Slater condition, or a positive-definite matrix
pencil), our result thus provides a novel extension to the so-called "hard
case" of the generalized trust region subproblem subject to the upper and the
lower level set of a quadratic function.Comment: 26 page
Interior Point Methods for Massive Support Vector Machines
We investigate the use of interior point methods for solving quadratic
programming problems with a small number of linear constraints where
the quadratic term consists of a low-rank update to a positive semi-de nite
matrix. Several formulations of the support vector machine t into this
category. An interesting feature of these particular problems is the vol-
ume of data, which can lead to quadratic programs with between 10 and
100 million variables and a dense Q matrix. We use OOQP, an object-
oriented interior point code, to solve these problem because it allows us
to easily tailor the required linear algebra to the application. Our linear
algebra implementation uses a proximal point modi cation to the under-
lying algorithm, and exploits the Sherman-Morrison-Woodbury formula
and the Schur complement to facilitate e cient linear system solution.
Since we target massive problems, the data is stored out-of-core and we
overlap computation and I/O to reduce overhead. Results are reported
for several linear support vector machine formulations demonstrating the
reliability and scalability of the method
Conic Optimization Theory: Convexification Techniques and Numerical Algorithms
Optimization is at the core of control theory and appears in several areas of
this field, such as optimal control, distributed control, system
identification, robust control, state estimation, model predictive control and
dynamic programming. The recent advances in various topics of modern
optimization have also been revamping the area of machine learning. Motivated
by the crucial role of optimization theory in the design, analysis, control and
operation of real-world systems, this tutorial paper offers a detailed overview
of some major advances in this area, namely conic optimization and its emerging
applications. First, we discuss the importance of conic optimization in
different areas. Then, we explain seminal results on the design of hierarchies
of convex relaxations for a wide range of nonconvex problems. Finally, we study
different numerical algorithms for large-scale conic optimization problems.Comment: 18 page
Fast interior point solution of quadratic programming problems arising from PDE-constrained optimization
Interior point methods provide an attractive class of approaches for solving linear, quadratic and nonlinear programming problems, due to their excellent efficiency and wide applicability. In this paper, we consider PDE-constrained optimization problems with bound constraints on the state and control variables, and their representation on the discrete level as quadratic programming problems. To tackle complex problems and achieve high accuracy in the solution, one is required to solve matrix systems of huge scale resulting from Newton iteration, and hence fast and robust methods for these systems are required. We present preconditioned iterative techniques for solving a number of these problems using Krylov subspace methods, considering in what circumstances one may predict rapid convergence of the solvers in theory, as well as the solutions observed from practical computations
- …