Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

Asymptotic analysis of semidefinite bounds for polynomial optimization and independent sets in geometric hypergraphs

The goal of a mathematical optimization problem is to maximize an objective (or minimize a cost) under a given set of rules, called constraints. Optimization has many applications, both in other areas of mathematics and in the real world. Unfortunately, some of the most interesting problems are also very hard to solve numerically. To work around this issue, one often considers relaxations: approximations of the original problem that are much easier to solve. Naturally, it is then important to understand how (in)accurate these relaxations are. This thesis consists of three parts, each covering a different method that uses semidefinite programming to approximate hard optimization problems. In Part 1 and Part 2, we consider two hierarchies of relaxations for polynomial optimization problems based on sums of squares. We show improved guarantees on the quality of Lasserre's measure-based hierarchy in a wide variety of settings (Part 1). We establish error bounds for the moment-SOS hierarchy in certain fundamental special cases. These bounds are much stronger than the ones obtained from existing, general results (Part 2). In Part 3, we generalize the celebrated Lovász theta number to (geometric) hypergraphs. We apply our generalization to formulate relaxations for a type of independent set problem in the hypersphere. These relaxations allow us to improve some results in Euclidean Ramsey theory

Mini-Workshop: Applied Koopmanism

Koopman and Perron–Frobenius operators are linear operators that encapsulate dynamics of nonlinear dynamical systems without loss of information. This is accomplished by embedding the dynamics into a larger infinite-dimensional space where the focus of study is shifted from trajectory curves to measurement functions evaluated along trajectories and densities of trajectories evolving in time. Operator-theoretic approach to dynamics shares many features with an optimization technique: the Lasserre moment–sums-of-squares (SOS) hierarchies, which was developed for numerically solving non-convex optimization problems with semialgebraic data. This technique embeds the optimization problem into a larger primal semidefinite programming (SDP) problem consisting of measure optimization over the set of globally optimal solutions, where measures are manipulated through their truncated moment sequences. The dual SDP problem uses SOS representations to certify bounds on the global optimum. This workshop highlighted the common threads between the operator-theoretic dynamical systems and moment–SOS hierarchies in optimization and explored the future directions where the synergy of the two techniques could yield results in fluid dynamics, control theory, optimization, and spectral theory

Some approximation schemes in polynomial optimization

Cette thèse est dédiée à l'étude de la hiérarchie moments-sommes-de-carrés, une famille de problèmes de programmation semi-définie en optimisation polynomiale, couramment appelée hiérarchie de Lasserre. Nous examinons différents aspects de ses propriétés et applications. Comme application de la hiérarchie, nous approchons certains objets potentiellement compliqués, comme l'abscisse polynomiale et les plans d'expérience optimaux sur des domaines semi-algébriques. L'application de la hiérarchie de Lasserre produit des approximations par des polynômes de degré fixé et donc de complexité bornée. En ce qui concerne la complexité de la hiérarchie elle-même, nous en construisons une modification pour laquelle un taux de convergence amélioré peut être prouvé. Un concept essentiel de la hiérarchie est l'utilisation des modules quadratiques et de leurs duaux pour appréhender de manière flexible le cône des polynômes positifs et le cône des moments. Nous poursuivons cette idée pour construire des approximations étroites d'ensembles semi-algébriques à l'aide de séparateurs polynomiaux.This thesis is dedicated to investigations of the moment-sums-of-squares hierarchy, a family of semidefinite programming problems in polynomial optimization, commonly called the Lasserre hierarchy. We examine different aspects of its properties and purposes. As applications of the hierarchy, we approximate some potentially complicated objects, namely the polynomial abscissa and optimal designs on semialgebraic domains. Applying the Lasserre hierarchy results in approximations by polynomials of fixed degree and hence bounded complexity. With regard to the complexity of the hierarchy itself, we construct a modification of it for which an improved convergence rate can be proved. An essential concept of the hierarchy is to use quadratic modules and their duals as a tractable characterization of the cone of positive polynomials and the moment cone, respectively. We exploit further this idea to construct tight approximations of semialgebraic sets with polynomial separators

Practical polynomial optimization through positivity certificates with and without denominators

Les certificats de positivité ou Positivstellens"atze fournissent des représentations de polynômes positifs sur des ensembles semialgébriques de basiques, c'est-à-dire des ensembles définis par un nombre fini d'inégalités polynomiales. Le célèbre Positivstellensatz de Putinar stipule que tout polynôme positif sur un ensemble semialgébrique basique fermé $S$ peut être écrit comme une combinaison pondérée linéaire des polynômes décrivant $S$, sous une certaine condition sur $S$ légèrement plus forte que la compacité. Lorsqu'il est écrit comme ceci, il devient évident que le polynôme est positif sur $S$, et donc cette description alternative fournit un certificat de positivité sur $S$. De plus, comme les poids polynomiaux impliqués dans le Positivstellensatz de Putinar sont des sommes de carrés (SOS), de tels certificats de positivité permettent de concevoir des relaxations convexes basées sur la programmation semidéfinie pour résoudre des problèmes d'optimisation polynomiale (POP) qui surviennent dans diverses applications réelles, par exemple dans la gestion des réseaux d'énergie et l'apprentissage automatique pour n'en citer que quelques unes. Développée à l'origine par Lasserre, la hiérarchie des relaxations semidéfinies basée sur le Positivstellensatz de Putinar est appelée la emph{hiérarchie Moment-SOS}. Dans cette thèse, nous proposons des méthodes d'optimisation polynomiale basées sur des certificats de positivité impliquant des poids SOS spécifiques, sans ou avec dénominateurs.Positivity certificates or Positivstellens"atze provide representations of polynomials positive on basic semialgebraic sets, i.e., sets defined by finitely many polynomial inequalities. The famous Putinar's Positivstellensatz states that every positive polynomial on a basic closed semialgebraic set $S$ can be written as a linear weighted combination of the polynomials describing $S$, under a certain condition on $S$ slightly stronger than compactness. When written in this it becomes obvious that the polynomial is positive on $S$, and therefore this alternative description provides a certificate of positivity on $S$. Moreover, as the polynomial weights involved in Putinar's Positivstellensatz are sums of squares (SOS), such Positivity certificates enable to design convex relaxations based on semidefinite programming to solve polynomial optimization problems (POPs) that arise in various real-life applications, e.g., in management of energy networks and machine learning to cite a few. Originally developed by Lasserre, the hierarchy of semidefinite relaxations based on Putinar's Positivstellensatz is called the emph{Moment-SOS hierarchy}. In this thesis, we provide polynomial optimization methods based on positivity certificates involving specific SOS weights, without or with denominators

Convergence rates for sums-of-squares hierarchies with correlative sparsity

This work derives upper bounds on the convergence rate of the moment-sum-of-squares hierarchy with correlative sparsity for global minimization of polynomials on compact basic semialgebraic sets. The main conclusion is that both sparse hierarchies based on the Schm\"udgen and Putinar Positivstellens\"atze enjoy a polynomial rate of convergence that depends on the size of the largest clique in the sparsity graph but not on the ambient dimension. Interestingly, the sparse bounds outperform the best currently available bounds for the dense hierarchy when the maximum clique size is sufficiently small compared to the ambient dimension and the performance is measured by the running time of an interior point method required to obtain a bound on the global minimum of a given accuracy.Comment: 23 page

Moment-sum-of-squares hierarchies for set approximation and optimal control

This thesis uses the idea of lifting (or embedding) a nonlinear controlled dynamical system into an infinite-dimensional space of measures where this system is equivalently described by a linear equation. This equation and problems involving it are subsequently approximated using well-known moment-sum-of-squares hierarchies. First, we address the problems of region of attraction, reachable set and maximum controlled invariant set computation, where we provide a characterization of these sets as an infinite-dimensional linear program in the cone of nonnegative measures and we describe a hierarchy of finite-dimensional semidefinite-programming (SDP) hierarchies providing a converging sequence of outer approximations to these sets. Next, we treat the problem of optimal feedback controller design under state and input constraints. We provide a hierarchy of SDPs yielding an asymptotically optimal sequence of rational feedback controllers. In addition, we describe hierarchies of SDPs yielding approximations to the value function attained by any given rational controller, from below and from above, as well as a hierarchy of SDPs providing approximations from below to the optimal value function, hence obtaining performance certificates for the designed controllers as well as for any given rational controller. Finally, we describe a method to verify properties of a closed loop interconnection of a nonlinear dynamical system and an optimization-based controller (e.g., a model predictive controller) for deterministic and stochastic nonlinear dynamical systems. Properties such as global stability, the $\ell_2$ gain or performance with respect to a given infinite-horizon cost function can be certified. The methods presented are easy to implement using freely available software packages and are documented by a number of numerical examples

Stokes, Gibbs and volume computation of semi-algebraic sets

We consider the problem of computing the Lebesgue volume of compact basic semi-algebraic sets. In full generality, it can be approximated as closely as desired by a converging hierarchy of upper bounds obtained by applying the Moment-SOS (sums of squares) methodology to a certain innite-dimensional linear program (LP). At each step one solves a semidenite relaxation of the LP which involves pseudo-moments up to a certain degree. Its dual computes a polynomial of same degree which approximates from above the discon-tinuous indicator function of the set, hence with a typical Gibbs phenomenon which results in a slow convergence of the associated numerical scheme. Drastic improvements have been observed by introducing in the initial LP additional linear moment constraints obtained from a certain application of Stokes' theorem for integration on the set. However and so far there was no rationale to explain this behavior. We provide a rened version of this extended LP formulation. When the set is the smooth super-level set of a single polynomial, we show that the dual of this rened LP has an optimal solution which is a continuous function. Therefore in this dual one now approximates a continuous function by a polynomial, hence with no Gibbs phenomenon, which explains and improves the already observed drastic acceleration of the convergence of the hierarchy. Interestingly, the technique of proof involves classical results on Poisson's partial dierential equation (PDE)

Sum-of-squares hierarchies for binary polynomial optimization

We consider the sum-of-squares hierarchy of approximations for the problem of minimizing a polynomial $f$ over the boolean hypercube $\mathbb{B}^{n}=\{0,1\}^n$. This hierarchy provides for each integer $r \in \mathbb{N}$ a lower bound $f_{(r)}$ on the minimum $f_{\min}$ of $f$, given by the largest scalar $\lambda$ for which the polynomial $f - \lambda$ is a sum-of-squares on $\mathbb{B}^{n}$ with degree at most $2r$. We analyze the quality of these bounds by estimating the worst-case error $f_{\min} - f_{(r)}$ in terms of the least roots of the Krawtchouk polynomials. As a consequence, for fixed $t \in [0, 1/2]$, we can show that this worst-case error in the regime $r \approx t \cdot n$ is of the order $1/2 - \sqrt{t(1-t)}$ as $n$ tends to $\infty$. Our proof combines classical Fourier analysis on $\mathbb{B}^{n}$ with the polynomial kernel technique and existing results on the extremal roots of Krawtchouk polynomials. This link to roots of orthogonal polynomials relies on a connection between the hierarchy of lower bounds $f_{(r)}$ and another hierarchy of upper bounds $f^{(r)}$, for which we are also able to establish the same error analysis. Our analysis extends to the minimization of a polynomial over the $q$-ary cube $(\mathbb{Z}/q\mathbb{Z})^{n}$.Comment: 23 pages, 1 figure. Fixed a typo in Theorem 1 and Theorem