Improving Efficiency and Scalability of Sum of Squares Optimization: Recent Advances and Limitations

It is well-known that any sum of squares (SOS) program can be cast as a semidefinite program (SDP) of a particular structure and that therein lies the computational bottleneck for SOS programs, as the SDPs generated by this procedure are large and costly to solve when the polynomials involved in the SOS programs have a large number of variables and degree. In this paper, we review SOS optimization techniques and present two new methods for improving their computational efficiency. The first method leverages the sparsity of the underlying SDP to obtain computational speed-ups. Further improvements can be obtained if the coefficients of the polynomials that describe the problem have a particular sparsity pattern, called chordal sparsity. The second method bypasses semidefinite programming altogether and relies instead on solving a sequence of more tractable convex programs, namely linear and second order cone programs. This opens up the question as to how well one can approximate the cone of SOS polynomials by second order representable cones. In the last part of the paper, we present some recent negative results related to this question.Comment: Tutorial for CDC 201

A Complete Characterization of the Gap between Convexity and SOS-Convexity

Our first contribution in this paper is to prove that three natural sum of squares (sos) based sufficient conditions for convexity of polynomials, via the definition of convexity, its first order characterization, and its second order characterization, are equivalent. These three equivalent algebraic conditions, henceforth referred to as sos-convexity, can be checked by semidefinite programming whereas deciding convexity is NP-hard. If we denote the set of convex and sos-convex polynomials in $n$ variables of degree $d$ with $\tilde{C}_{n,d}$ and $\tilde{\Sigma C}_{n,d}$ respectively, then our main contribution is to prove that $\tilde{C}_{n,d}=\tilde{\Sigma C}_{n,d}$ if and only if $n=1$ or $d=2$ or $(n,d)=(2,4)$. We also present a complete characterization for forms (homogeneous polynomials) except for the case $(n,d)=(3,4)$ which is joint work with G. Blekherman and is to be published elsewhere. Our result states that the set $C_{n,d}$ of convex forms in $n$ variables of degree $d$ equals the set $\Sigma C_{n,d}$ of sos-convex forms if and only if $n=2$ or $d=2$ or $(n,d)=(3,4)$. To prove these results, we present in particular explicit examples of polynomials in $\tilde{C}_{2,6}\setminus\tilde{\Sigma C}_{2,6}$ and $\tilde{C}_{3,4}\setminus\tilde{\Sigma C}_{3,4}$ and forms in $C_{3,6}\setminus\Sigma C_{3,6}$ and $C_{4,4}\setminus\Sigma C_{4,4}$, and a general procedure for constructing forms in $C_{n,d+2}\setminus\Sigma C_{n,d+2}$ from nonnegative but not sos forms in $n$ variables and degree $d$. Although for disparate reasons, the remarkable outcome is that convex polynomials (resp. forms) are sos-convex exactly in cases where nonnegative polynomials (resp. forms) are sums of squares, as characterized by Hilbert.Comment: 25 pages; minor editorial revisions made; formal certificates for computer assisted proofs of the paper added to arXi

Finite convergence of sum-of-squares hierarchies for the stability number of a graph

We investigate a hierarchy of semidefinite bounds $\vartheta^{(r)}(G)$ for the stability number $\alpha(G)$ of a graph $G$, based on its copositive programming formulation and introduced by de Klerk and Pasechnik [SIAM J. Optim. 12 (2002), pp.875--892], who conjectured convergence to $\alpha(G)$ in $r=\alpha(G)-1$ steps. Even the weaker conjecture claiming finite convergence is still open. We establish links between this hierarchy and sum-of-squares hierarchies based on the Motzkin-Straus formulation of $\alpha(G)$, which we use to show finite convergence when $G$ is acritical, i.e., when $\alpha(G\setminus e)=\alpha(G)$ for all edges $e$ of $G$. This relies, in particular, on understanding the structure of the minimizers of Motzkin-Straus formulation and showing that their number is finite precisely when $G$ is acritical. As a byproduct we show that deciding whether a standard quadratic program has finitely many minimizers does not admit a polynomial-time algorithm unless P=NP.Comment: We removed the material from section 7 about rank 0 graphs which will be included in separate forthcoming wor

On the exactness of sum-of-squares approximations for the cone of 5 × 5 copositive matrices

We investigate the hierarchy of conic inner approximations Kn(r) (r∈N) for the copositive cone COPn, introduced by Parrilo (2000) [22]. It is known that COP4=K4(0) and that, while the union of the cones Kn(r) covers the interior of COPn, it does not cover the full cone COPn if n≥6. Here we investigate the remaining case n=5, where all extreme rays have been fully characterized by Hildebrand (2012) [12]. We show that the Horn matrix H and its positive diagonal scalings play an exceptional role among the extreme rays of COP5. We show that equality COP5=⋃r≥0K5(r) holds if and only if every positive diagonal scaling of H belongs to K5(r) for some r∈N. As a main ingredient for the proof, we introduce new Lasserre-type conic inner approximations for COPn, based on sums of squares of polynomials. We show their links to the cones Kn(r), and we use an optimization approach that permits to exploit finite convergence results on Lasserre hierarchy to show membership in the new cones

Exactness of Parrilo’s conic approximations for copositive matrices and associated low order bounds for the stability number of a graph

De Klerk and Pasechnik introduced in 2002 semidefinite bounds ϑ(r)(G)(r≥0) for the stability number α(G) of a graph G and conjectured their exactness at order r=α(G)−1. These bounds rely on the conic approximations K(r)n introduced by Parrilo in 2000 for the copositive cone COPn. A difficulty in the convergence analysis of the bounds is the bad behavior of Parrilo’s cones under adding a zero row/column: when applied to a matrix not in K(r)n this gives a matrix that does not lie in any of Parrilo’s cones, thereby showing that their union is a strict subset of the copositive cone for any n≥6. We investigate the graphs for which the bound of order r≤1 is exact: we algorithmically reduce testing exactness of ϑ(0) to acritical graphs, we characterize the critical graphs with ϑ(0) exact, and we exhibit graphs for which exactness of ϑ(1) is not preserved under adding an isolated node. This disproves a conjecture posed by Gvozdenović and Laurent in 2007, which, if true, would have implied the above conjecture by de Klerk and Pasechnik