Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

Optimal designs for rational function regression

We consider optimal non-sequential designs for a large class of (linear and nonlinear) regression models involving polynomials and rational functions with heteroscedastic noise also given by a polynomial or rational weight function. The proposed method treats D-, E-, A-, and $\Phi_p$-optimal designs in a unified manner, and generates a polynomial whose zeros are the support points of the optimal approximate design, generalizing a number of previously known results of the same flavor. The method is based on a mathematical optimization model that can incorporate various criteria of optimality and can be solved efficiently by well established numerical optimization methods. In contrast to previous optimization-based methods proposed for similar design problems, it also has theoretical guarantee of its algorithmic efficiency; in fact, the running times of all numerical examples considered in the paper are negligible. The stability of the method is demonstrated in an example involving high degree polynomials. After discussing linear models, applications for finding locally optimal designs for nonlinear regression models involving rational functions are presented, then extensions to robust regression designs, and trigonometric regression are shown. As a corollary, an upper bound on the size of the support set of the minimally-supported optimal designs is also found. The method is of considerable practical importance, with the potential for instance to impact design software development. Further study of the optimality conditions of the main optimization model might also yield new theoretical insights.Comment: 25 pages. Previous version updated with more details in the theory and additional example

Positive Forms and Stability of Linear Time-Delay Systems

We consider the problem of constructing Lyapunov functions for linear differential equations with delays. For such systems it is known that exponential stability implies the existence of a positive Lyapunov function which is quadratic on the space of continuous functions. We give an explicit parametrization of a sequence of finite-dimensional subsets of the cone of positive Lyapunov functions using positive semidefinite matrices. This allows stability analysis of linear time-delay systems to be formulated as a semidefinite program.Comment: journal version, 14 page

Generating Non-Linear Interpolants by Semidefinite Programming

Interpolation-based techniques have been widely and successfully applied in the verification of hardware and software, e.g., in bounded-model check- ing, CEGAR, SMT, etc., whose hardest part is how to synthesize interpolants. Various work for discovering interpolants for propositional logic, quantifier-free fragments of first-order theories and their combinations have been proposed. However, little work focuses on discovering polynomial interpolants in the literature. In this paper, we provide an approach for constructing non-linear interpolants based on semidefinite programming, and show how to apply such results to the verification of programs by examples.Comment: 22 pages, 4 figure

Inverse polynomial optimization

We consider the inverse optimization problem associated with the polynomial program f^*=\min \{f(x): x\in K\}$and a given current feasible solution$y\in K$. We provide a systematic numerical scheme to compute an inverse optimal solution. That is, we compute a polynomial$\tilde{f}$(which may be of same degree as$f$if desired) with the following properties: (a)$y$is a global minimizer of$\tilde{f}$on$K$with a Putinar's certificate with an a priori degree bound$d$fixed, and (b),$\tilde{f}$minimizes$\Vert f-\tilde{f}\Vert$(which can be the$\ell_1$,$\ell_2$or$\ell_\infty$-norm of the coefficients) over all polynomials with such properties. Computing$\tilde{f}_d$reduces to solving a semidefinite program whose optimal value also provides a bound on how far is$f(\y)$from the unknown optimal value$f^*$. The size of the semidefinite program can be adapted to the computational capabilities available. Moreover, if one uses the$\ell_1$-norm, then$\tilde{f}$ takes a simple and explicit canonical form. Some variations are also discussed.Comment: 25 pages; to appear in Math. Oper. Res; Rapport LAAS no. 1114

Bounding stationary averages of polynomial diffusions via semidefinite programming

We introduce an algorithm based on semidefinite programming that yields increasing (resp. decreasing) sequences of lower (resp. upper) bounds on polynomial stationary averages of diffusions with polynomial drift vector and diffusion coefficients. The bounds are obtained by optimising an objective, determined by the stationary average of interest, over the set of real vectors defined by certain linear equalities and semidefinite inequalities which are satisfied by the moments of any stationary measure of the diffusion. We exemplify the use of the approach through several applications: a Bayesian inference problem; the computation of Lyapunov exponents of linear ordinary differential equations perturbed by multiplicative white noise; and a reliability problem from structural mechanics. Additionally, we prove that the bounds converge to the infimum and supremum of the set of stationary averages for certain SDEs associated with the computation of the Lyapunov exponents, and we provide numerical evidence of convergence in more general settings

Backstepping PDE Design: A Convex Optimization Approach

Abstract\u2014Backstepping design for boundary linear PDE is formulated as a convex optimization problem. Some classes of parabolic PDEs and a first-order hyperbolic PDE are studied, with particular attention to non-strict feedback structures. Based on the compactness of the Volterra and Fredholm-type operators involved, their Kernels are approximated via polynomial functions. The resulting Kernel-PDEs are optimized using Sumof- Squares (SOS) decomposition and solved via semidefinite programming, with sufficient precision to guarantee the stability of the system in the L2-norm. This formulation allows optimizing extra degrees of freedom where the Kernel-PDEs are included as constraints. Uniqueness and invertibility of the Fredholm-type transformation are proved for polynomial Kernels in the space of continuous functions. The effectiveness and limitations of the approach proposed are illustrated by numerical solutions of some Kernel-PDEs