Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

Copositive certificates of non-negativity for polynomials on semialgebraic sets

A certificate of non-negativity is a way to write a given function so that its non-negativity becomes evident. Certificates of non-negativity are fundamental tools in optimization, and they underlie powerful algorithmic techniques for various types of optimization problems. We propose certificates of non-negativity of polynomials based on copositive polynomials. The certificates we obtain are valid for generic semialgebraic sets and have a fixed small degree, while commonly used sums-of-squares (SOS) certificates are guaranteed to be valid only for compact semialgebraic sets and could have large degree. Optimization over the cone of copositive polynomials is not tractable, but this cone has been well studied. The main benefit of our copositive certificates of non-negativity is their ability to translate results known exclusively for copositive polynomials to more general semialgebraic sets. In particular, we show how to use copositive polynomials to construct structured (e.g., sparse) certificates of non-negativity, even for unstructured semialgebraic sets. Last but not least, copositive certificates can be used to obtain not only hierarchies of tractable lower bounds, but also hierarchies of tractable upper bounds for polynomial optimization problems.Comment: 27 pages, 1 figur

The many faces of positivity to approximate structured optimization problems

The PhD dissertation proposes tractable linear and semidefinite relaxations for optimization problems that are hard to solve and approximate, such as polynomial or copositive problems. To do this, we exploit the structure and inherent symmetry of these problems. The thesis consists of five essays devoted to distinct problems. First, we consider the kissing number problem. The kissing number is the maximum number of non-overlapping unit spheres that can simultaneously touch another unit sphere, in n-dimensional space. In chapter two we construct a new hierarchy of upper bounds on the kissing number. To implement the hierarchy, in chapter three we propose two generalizations of Schoenberg's theorem on positive definite kernels. In the fourth chapter, we derive new certificates of non-negativity of polynomials on generic sets defined by polynomial equalities and inequalities. These certificates are based on copositive polynomials and allow obtaining new upper and lower bounds for polynomial optimization problems. In chapter five, for any given graph we look for the largest k-colorable subgraph; that is, the induced subgraph that can be colored in k colors such that no two adjacent vertices have the same color. We obtain several new semidefinite programming relaxations to this problem. In the final sixth chapter, we consider the problem of allocating tasks to unrelated parallel selfish machines to minimize the time to complete all the tasks. For this problem, we suggest new upper and lower bounds on the best approximation ratio of a class of truthful task allocation algorithms

Practical polynomial optimization through positivity certificates with and without denominators

Les certificats de positivité ou Positivstellens"atze fournissent des représentations de polynômes positifs sur des ensembles semialgébriques de basiques, c'est-à-dire des ensembles définis par un nombre fini d'inégalités polynomiales. Le célèbre Positivstellensatz de Putinar stipule que tout polynôme positif sur un ensemble semialgébrique basique fermé $S$ peut être écrit comme une combinaison pondérée linéaire des polynômes décrivant $S$, sous une certaine condition sur $S$ légèrement plus forte que la compacité. Lorsqu'il est écrit comme ceci, il devient évident que le polynôme est positif sur $S$, et donc cette description alternative fournit un certificat de positivité sur $S$. De plus, comme les poids polynomiaux impliqués dans le Positivstellensatz de Putinar sont des sommes de carrés (SOS), de tels certificats de positivité permettent de concevoir des relaxations convexes basées sur la programmation semidéfinie pour résoudre des problèmes d'optimisation polynomiale (POP) qui surviennent dans diverses applications réelles, par exemple dans la gestion des réseaux d'énergie et l'apprentissage automatique pour n'en citer que quelques unes. Développée à l'origine par Lasserre, la hiérarchie des relaxations semidéfinies basée sur le Positivstellensatz de Putinar est appelée la emph{hiérarchie Moment-SOS}. Dans cette thèse, nous proposons des méthodes d'optimisation polynomiale basées sur des certificats de positivité impliquant des poids SOS spécifiques, sans ou avec dénominateurs.Positivity certificates or Positivstellens"atze provide representations of polynomials positive on basic semialgebraic sets, i.e., sets defined by finitely many polynomial inequalities. The famous Putinar's Positivstellensatz states that every positive polynomial on a basic closed semialgebraic set $S$ can be written as a linear weighted combination of the polynomials describing $S$, under a certain condition on $S$ slightly stronger than compactness. When written in this it becomes obvious that the polynomial is positive on $S$, and therefore this alternative description provides a certificate of positivity on $S$. Moreover, as the polynomial weights involved in Putinar's Positivstellensatz are sums of squares (SOS), such Positivity certificates enable to design convex relaxations based on semidefinite programming to solve polynomial optimization problems (POPs) that arise in various real-life applications, e.g., in management of energy networks and machine learning to cite a few. Originally developed by Lasserre, the hierarchy of semidefinite relaxations based on Putinar's Positivstellensatz is called the emph{Moment-SOS hierarchy}. In this thesis, we provide polynomial optimization methods based on positivity certificates involving specific SOS weights, without or with denominators

Global minimization of polynomial integral functionals

We describe a `discretize-then-relax' strategy to globally minimize integral functionals over functions $u$ in a Sobolev space satisfying prescribed Dirichlet boundary conditions. The strategy applies whenever the integral functional depends polynomially on $u$ and its derivatives, even if it is nonconvex. The `discretize' step uses a bounded finite-element scheme to approximate the integral minimization problem with a convergent hierarchy of polynomial optimization problems over a compact feasible set, indexed by the decreasing size $h$ of the finite-element mesh. The `relax' step employs sparse moment-SOS relaxations to approximate each polynomial optimization problem with a hierarchy of convex semidefinite programs, indexed by an increasing relaxation order $\omega$. We prove that, as $\omega\to\infty$ and $h\to 0$, solutions of such semidefinite programs provide approximate minimizers that converge in $L^p$ to the global minimizer of the original integral functional if this is unique. We also report computational experiments that show our numerical strategy works well even when technical conditions required by our theoretical analysis are not satisfied.Comment: 22 pages, 9 figure