Handelman 's hierarchy for the maximum stable set problem.

The maximum stable set problem is a well-known NP-hard problem in combinatorial optimization, which can be formulated as the maximization of a quadratic square-free polynomial over the (Boolean) hypercube. We investigate a hierarchy of linear programming relaxations for this problem, based on a result of Handelman showing that a positive polynomial over a polytope with non-empty interior can be represented as conic combination of products of the linear constraints defining the polytope. We relate the rank of Handelman's hierarchy with structural properties of graphs. In particular we show a relation to fractional clique covers which we use to upper bound the Handelman rank for perfect graphs and determine its exact value in the vertex-transitive case. Moreover we show two upper bounds on the Handelman rank in terms of the (fractional) stability number of the graph and compute the Handelman rank for several classes of graphs including odd cycles and wheels and their complements. We also point out links to several other linear and semidefinite programming hierarchies

Degree bounds for Putinar's Positivstellensatz on the hypercube

The Positivstellens\"atze of Putinar and Schm\"udgen show that any polynomial $f$ positive on a compact semialgebraic set can be represented using sums of squares. Recently, there has been large interest in proving effective versions of these results, namely to show bounds on the required degree of the sums of squares in such representations. These effective Positivstellens\"atze have direct implications for the convergence rate of the celebrated moment-SOS hierarchy in polynomial optimization. In this paper, we restrict to the fundamental case of the hypercube $\mathrm{B}^{n} = [-1, 1]^n$. We show an upper degree bound for Putinar-type representations on $\mathrm{B}^{n}$ of the order $O(f_{\max}/f_{\min})$, where $f_{\max}$, $f_{\min}$ are the maximum and minimum of $f$ on $\mathrm{B}^{n}$, respectively. Previously, specialized results of this kind were available only for Schm\"udgen-type representations and not for Putinar-type ones. Complementing this upper degree bound, we show a lower degree bound in $\Omega(\sqrt[8]{f_{\max}/f_{\min}})$. This is the first lower bound for Putinar-type representations on a semialgebraic set with nonempty interior described by a standard set of inequalities.Comment: Final version. Improved presentation and included a more detailed comparison with Bach and Rudi, Exponential Convergence of Sum-of-Squares Hierarchies for Trigonometric Polynomials, SIAM Opt., 202

Improved convergence rates for Lasserre-type hierarchies of upper bounds for box-constrained polynomial optimization

We consider the problem of minimizing a given multivariate polynomial f over the hypercube [-1,1]^n. An idea, introduced by Lasserre, is to find a probability distribution on the hypercube with polynomial density function h (of given degree r) that minimizes the expectation of f over the hypercube with respect to this probability distribution. It is known that, for the Lebesgue measure one may show an error bound in 1/sqrt{r} if h is a sum-of-squares density, and an error bound in 1/r if h is the density of a beta distribution. In this paper, we show another probability distribution that permits to show an error bound in 1/r^2 when selecting a density function h with a Schmuedgen-type sum-of-squares decomposition. The convergence rate analysis relies on the theory of polynomial kernels, and in particular on Jackson kernels. We also show that the resulting upper bounds may be computed as generalized eigenvalue problems, as is also the case for sum-of-squares densitie

Decomposition Methods for Nonlinear Optimization and Data Mining

We focus on two central themes in this dissertation. The first one is on decomposing polytopes and polynomials in ways that allow us to perform nonlinear optimization. We start off by explaining important results on decomposing a polytope into special polyhedra. We use these decompositions and develop methods for computing a special class of integrals exactly. Namely, we are interested in computing the exact value of integrals of polynomial functions over convex polyhedra. We present prior work and new extensions of the integration algorithms. Every integration method we present requires that the polynomial has a special form. We explore two special polynomial decomposition algorithms that are useful for integrating polynomial functions. Both polynomial decompositions have strengths and weaknesses, and we experiment with how to practically use them. After developing practical algorithms and efficient software tools for integrating a polynomial over a polytope, we focus on the problem of maximizing a polynomial function over the continuous domain of a polytope. This maximization problem is NP-hard, but we develop approximation methods that run in polynomial time when the dimension is fixed. Moreover, our algorithm for approximating the maximum of a polynomial over a polytope is related to integrating the polynomial over the polytope. We show how the integration methods can be used for optimization. The second central topic in this dissertation is on problems in data science. We first consider a heuristic for mixed-integer linear optimization. We show how many practical mixed-integer linear have a special substructure containing set partition constraints. We then describe a nice data structure for finding feasible zero-one integer solutions to systems of set partition constraints. Finally, we end with an applied project using data science methods in medical research.Comment: PHD Thesis of Brandon Dutr

Asymptotic analysis of semidefinite bounds for polynomial optimization and independent sets in geometric hypergraphs

The goal of a mathematical optimization problem is to maximize an objective (or minimize a cost) under a given set of rules, called constraints. Optimization has many applications, both in other areas of mathematics and in the real world. Unfortunately, some of the most interesting problems are also very hard to solve numerically. To work around this issue, one often considers relaxations: approximations of the original problem that are much easier to solve. Naturally, it is then important to understand how (in)accurate these relaxations are. This thesis consists of three parts, each covering a different method that uses semidefinite programming to approximate hard optimization problems. In Part 1 and Part 2, we consider two hierarchies of relaxations for polynomial optimization problems based on sums of squares. We show improved guarantees on the quality of Lasserre's measure-based hierarchy in a wide variety of settings (Part 1). We establish error bounds for the moment-SOS hierarchy in certain fundamental special cases. These bounds are much stronger than the ones obtained from existing, general results (Part 2). In Part 3, we generalize the celebrated Lovász theta number to (geometric) hypergraphs. We apply our generalization to formulate relaxations for a type of independent set problem in the hypersphere. These relaxations allow us to improve some results in Euclidean Ramsey theory

Practical polynomial optimization through positivity certificates with and without denominators

Les certificats de positivité ou Positivstellens"atze fournissent des représentations de polynômes positifs sur des ensembles semialgébriques de basiques, c'est-à-dire des ensembles définis par un nombre fini d'inégalités polynomiales. Le célèbre Positivstellensatz de Putinar stipule que tout polynôme positif sur un ensemble semialgébrique basique fermé $S$ peut être écrit comme une combinaison pondérée linéaire des polynômes décrivant $S$, sous une certaine condition sur $S$ légèrement plus forte que la compacité. Lorsqu'il est écrit comme ceci, il devient évident que le polynôme est positif sur $S$, et donc cette description alternative fournit un certificat de positivité sur $S$. De plus, comme les poids polynomiaux impliqués dans le Positivstellensatz de Putinar sont des sommes de carrés (SOS), de tels certificats de positivité permettent de concevoir des relaxations convexes basées sur la programmation semidéfinie pour résoudre des problèmes d'optimisation polynomiale (POP) qui surviennent dans diverses applications réelles, par exemple dans la gestion des réseaux d'énergie et l'apprentissage automatique pour n'en citer que quelques unes. Développée à l'origine par Lasserre, la hiérarchie des relaxations semidéfinies basée sur le Positivstellensatz de Putinar est appelée la emph{hiérarchie Moment-SOS}. Dans cette thèse, nous proposons des méthodes d'optimisation polynomiale basées sur des certificats de positivité impliquant des poids SOS spécifiques, sans ou avec dénominateurs.Positivity certificates or Positivstellens"atze provide representations of polynomials positive on basic semialgebraic sets, i.e., sets defined by finitely many polynomial inequalities. The famous Putinar's Positivstellensatz states that every positive polynomial on a basic closed semialgebraic set $S$ can be written as a linear weighted combination of the polynomials describing $S$, under a certain condition on $S$ slightly stronger than compactness. When written in this it becomes obvious that the polynomial is positive on $S$, and therefore this alternative description provides a certificate of positivity on $S$. Moreover, as the polynomial weights involved in Putinar's Positivstellensatz are sums of squares (SOS), such Positivity certificates enable to design convex relaxations based on semidefinite programming to solve polynomial optimization problems (POPs) that arise in various real-life applications, e.g., in management of energy networks and machine learning to cite a few. Originally developed by Lasserre, the hierarchy of semidefinite relaxations based on Putinar's Positivstellensatz is called the emph{Moment-SOS hierarchy}. In this thesis, we provide polynomial optimization methods based on positivity certificates involving specific SOS weights, without or with denominators

Sum-of-squares representations for copositive matrices and independent sets in graphs

A polynomial optimization problem asks for minimizing a polynomial function (cost) given a set of constraints (rules) represented by polynomial inequalities and equations. Many hard problems in combinatorial optimization and applications in operations research can be naturally encoded as polynomial optimization problems. A common approach for addressing such computationally hard problems is by considering variations of the original problem that give an approximate solution, and that can be solved efficiently. One such approach for attacking hard combinatorial problems and, more generally, polynomial optimization problems, is given by the so-called sum-of-squares approximations. This thesis focuses on studying whether these approximations find the optimal solution of the original problem.We investigate this question in two main settings: 1) Copositive programs and 2) parameters dealing with independent sets in graphs. Among our main new results, we characterize the matrix sizes for which sum-of-squares approximations are able to capture all copositive matrices. In addition, we show finite convergence of the sums-of-squares approximations for maximum independent sets in graphs based on their continuous copositive reformulations. We also study sum-of-squares approximations for parameters asking for maximum balanced independent sets in bipartite graphs. In particular, we find connections with the Lovász theta number and we design eigenvalue bounds for several related parameters when the graphs satisfy some symmetry properties.<br/

Some approximation schemes in polynomial optimization

Cette thèse est dédiée à l'étude de la hiérarchie moments-sommes-de-carrés, une famille de problèmes de programmation semi-définie en optimisation polynomiale, couramment appelée hiérarchie de Lasserre. Nous examinons différents aspects de ses propriétés et applications. Comme application de la hiérarchie, nous approchons certains objets potentiellement compliqués, comme l'abscisse polynomiale et les plans d'expérience optimaux sur des domaines semi-algébriques. L'application de la hiérarchie de Lasserre produit des approximations par des polynômes de degré fixé et donc de complexité bornée. En ce qui concerne la complexité de la hiérarchie elle-même, nous en construisons une modification pour laquelle un taux de convergence amélioré peut être prouvé. Un concept essentiel de la hiérarchie est l'utilisation des modules quadratiques et de leurs duaux pour appréhender de manière flexible le cône des polynômes positifs et le cône des moments. Nous poursuivons cette idée pour construire des approximations étroites d'ensembles semi-algébriques à l'aide de séparateurs polynomiaux.This thesis is dedicated to investigations of the moment-sums-of-squares hierarchy, a family of semidefinite programming problems in polynomial optimization, commonly called the Lasserre hierarchy. We examine different aspects of its properties and purposes. As applications of the hierarchy, we approximate some potentially complicated objects, namely the polynomial abscissa and optimal designs on semialgebraic domains. Applying the Lasserre hierarchy results in approximations by polynomials of fixed degree and hence bounded complexity. With regard to the complexity of the hierarchy itself, we construct a modification of it for which an improved convergence rate can be proved. An essential concept of the hierarchy is to use quadratic modules and their duals as a tractable characterization of the cone of positive polynomials and the moment cone, respectively. We exploit further this idea to construct tight approximations of semialgebraic sets with polynomial separators

Stability and weight smoothing in CMAC neural networks

Although the CMAC (Cerebellar Model Articulation Controller) neural network has been successfully used in control systems for many years, its property of local generalization, the availability of trained information for network responses at adjacent untrained locations, although responsible for the networks rapid learning and efficient implementation, results in network responses that is, when trained with sparse or widely spaced training data, spiky in nature even when the underlying function being learned is quite smooth. Since the derivative of such a network response can vary widely, the CMAC\u27s usefulness for solving optimization problems as well as for certain other control system applications can be severely limited. This dissertation presents the CMAC algorithm in sufficient detail to explore its strengths and weaknesses. Its properties of information generalization and storage are discussed and comparisons are made with other neural network algorithms and with other adaptive control algorithms. A synopsis of the development of the fields of neural networks and adaptive control is included to lend historical perspective. A stability analysis of the CMAC algorithm for open-loop function learning is developed. This stability analysis casts the function learning problem as a unique implementation of the model reference structure and develops a Lyapunov function to prove convergence of the CMAC to the target model. A new CMAC learning rule is developed by treating the CMAC as a set of simultaneous equations in a constrained optimization problem and making appropriate choices for the weight penalty matrix in the cost equation. This dissertation then presents a new CMAC learning algorithm which has the property of weight smoothing to improve generalization, function approximation in partially trained networks and the partial derivatives of learned functions. This new learning algorithm is significant in that it derives from an optimum solution and demonstrates a dramatic performance improvement for function learning in the presence of widely spaced training data. Developed from a completely unique analytical direction, this algorithm represents a coupling and extension of single- and multi-resolution CMAC algorithms developed by other researchers. The insights derived from the analysis of the optimum solution and the resulting new learning rules are discussed and suggestions for future work are presented