Exploiting ideal-sparsity in the generalized moment problem with application to matrix factorization ranks

We explore a new type of sparsity for the generalized moment problem (GMP) that we call ideal-sparsity. This sparsity exploits the presence of equality constraints requiring the measure to be supported on the variety of an ideal generated by bilinear monomials modeled by an associated graph. We show that this enables an equivalent sparse reformulation of the GMP, where the single (high dimensional) measure variable is replaced by several (lower-dimensional) measure variables supported on the maximal cliques of the graph. We explore the resulting hierarchies of moment-based relaxations for the original dense formulation of GMP and this new, equivalent ideal-sparse reformulation, when applied to the problem of bounding nonnegative- and completely positive matrix factorization ranks. We show that the ideal-sparse hierarchies provide bounds that are at least as good (and often tighter) as those obtained from the dense hierarchy. This is in sharp contrast to the situation when exploiting correlative sparsity, as is most common in the literature, where the resulting bounds are weaker than the dense bounds. Moreover, while correlative sparsity requires the underlying graph to be chordal, no such assumption is needed for ideal-sparsity. Numerical results show that the ideal-sparse bounds are often tighter and much faster to compute than their dense analogs.Comment: 36 pages, 3 figure

Some approximation schemes in polynomial optimization

Cette thèse est dédiée à l'étude de la hiérarchie moments-sommes-de-carrés, une famille de problèmes de programmation semi-définie en optimisation polynomiale, couramment appelée hiérarchie de Lasserre. Nous examinons différents aspects de ses propriétés et applications. Comme application de la hiérarchie, nous approchons certains objets potentiellement compliqués, comme l'abscisse polynomiale et les plans d'expérience optimaux sur des domaines semi-algébriques. L'application de la hiérarchie de Lasserre produit des approximations par des polynômes de degré fixé et donc de complexité bornée. En ce qui concerne la complexité de la hiérarchie elle-même, nous en construisons une modification pour laquelle un taux de convergence amélioré peut être prouvé. Un concept essentiel de la hiérarchie est l'utilisation des modules quadratiques et de leurs duaux pour appréhender de manière flexible le cône des polynômes positifs et le cône des moments. Nous poursuivons cette idée pour construire des approximations étroites d'ensembles semi-algébriques à l'aide de séparateurs polynomiaux.This thesis is dedicated to investigations of the moment-sums-of-squares hierarchy, a family of semidefinite programming problems in polynomial optimization, commonly called the Lasserre hierarchy. We examine different aspects of its properties and purposes. As applications of the hierarchy, we approximate some potentially complicated objects, namely the polynomial abscissa and optimal designs on semialgebraic domains. Applying the Lasserre hierarchy results in approximations by polynomials of fixed degree and hence bounded complexity. With regard to the complexity of the hierarchy itself, we construct a modification of it for which an improved convergence rate can be proved. An essential concept of the hierarchy is to use quadratic modules and their duals as a tractable characterization of the cone of positive polynomials and the moment cone, respectively. We exploit further this idea to construct tight approximations of semialgebraic sets with polynomial separators

Polynomial optimization: matrix factorization ranks, portfolio selection, and queueing theory

Inspired by Leonhard Euler’s belief that every event in the world can be understood in terms of maximizing or minimizing a specific quantity, this thesis delves into the realm of mathematical optimization. The thesis is divided into four parts, with optimization acting as the unifying thread. Part 1 introduces a particular class of optimization problems called generalized moment problems (GMPs) and explores the moment method, a powerful tool used to solve GMPs. We introduce the new concept of ideal sparsity, a technique that aids in solving GMPs by improving the bounds of their associated hierarchy of semidefinite programs. Part 2 focuses on matrix factorization ranks, in particular, the nonnegative rank, the completely positive rank, and the separable rank. These ranks are extensively studied using the moment method, and ideal sparsity is applied (whenever possible) to enhance the bounds on these ranks and speed-up their computation. Part 3 centers around portfolio optimization and the mean-variance-skewness kurtosis (MVSK) problem. Multi-objective optimization techniques are employed to uncover Pareto optimal solutions to the MVSK problem. We show that most linear scalarizations of the MVSK problem result in specific convex polynomial optimization problems which can be solved efficiently. Part 4 explores hypergraph-based polynomials emerging from queueing theory in the setting of parallel-server systems with job redundancy policies. By exploiting the symmetry inherent in the polynomials and some classical results on matrix algebras, the convexity of these polynomials is demonstrated, thereby allowing us to prove that the polynomials attain their optima at the barycenter of the simplex.<br/

Trace-positive polynomials, sums of hermitian squares and the tracial moment problem

A polynomial ▫$f$▫ in non-commuting variables is trace-positive if the trace of ▫$f(underline{A})$▫ is positive for all tuples ▫$underline{A}$▫ of symmetric matrices of the same size. The investigation of trace-positive polynomials and of the question of when they can be written as a sum of hermitian squares and commutators of polynomials are motivated by their connection to two famous conjectures: The BMV conjecture from statistical quantum mechanics and the embedding conjecture of Alain Connes concerning von Neumann algebras. First, results on the question of when a trace-positive polynomial in two non-commuting variables can be written as a sum of hermitian squares and commutators are presented. For instance, any bivariate trace-positive polynomial of degree at most four has such a representation, whereas this is false in general if the degree is at least six. This is in perfect analogy to Hilbert\u27s results from the commutative context. Further, a partial answer to the Lieb-Seiringer formulation of the BMV conjecture is given by presenting some concrete representations of the polynomials ▫$S_{m,4}(X^2Y^2)$▫ as a sum of hermitian squares and commutators. The second part of this work deals with the tracial moment problem. That is, how can one describe sequences of real numbers that are given by tracial moments of a probability measure on symmetric matrices of a fixed size. The truncated tracial moment problem, where one considers only finite sequences, as well as the tracial analog of the ▫$K$▫-moment problem are also investigated. Several results from the classical moment problem in Functional Analysis can be transferred to this context. For instance, a tracial analog of Haviland\u27s theorem holds: A traciallinear functional ▫$L$▫ is given by the tracial moments of a positive Borel measure on symmetric matrices of a fixed size s if and only if ▫$L$▫ takes only positive values on all polynomials which are trace-positive on all tuples of symmetric ▫$s times s$▫-matrices. This result uses tracial versions of the results of Fialkow and Nie on positive extensions of truncated sequences. Further, tracial analogs of results of Stochel and of Bayer and Teichmann are given. Defining a tracial Hankel matrix in analogy to the Hankel matrix in the classical moment problem, the results of Curto and Fialkow concerning sequences with Hankel matrices of finite rank or Hankel matrices of finite size which admit a flat extension also hold true in the tracial context. Finally, a relaxation for trace-minimization of polynomials using sums of hermitian squares and commutators is proposed. While this relaxation is not always exact, the tracial analogs of the results of Curto and Fialkow give a sufficient condition for the exactness of this relaxation

Low-Complexity Modeling for Visual Data: Representations and Algorithms

With increasing availability and diversity of visual data generated in research labs and everyday life, it is becoming critical to develop disciplined and practical computation tools for such data. This thesis focuses on the low complexity representations and algorithms for visual data, in light of recent theoretical and algorithmic developments in high-dimensional data analysis. We first consider the problem of modeling a given dataset as superpositions of basic motifs. This model arises from several important applications, including microscopy image analysis, neural spike sorting and image deblurring. This motif-finding problem can be phrased as "short-and-sparse" blind deconvolution, in which the goal is to recover a short convolution kernel from its convolution with a sparse and random spike train. We normalize the convolution kernel to have unit Frobenius norm and then cast the blind deconvolution problem as a nonconvex optimization problem over the kernel sphere. We demonstrate that (i) in a certain region of the sphere, every local optimum is close to some shift truncation of the ground truth, when the activation spike is sufficiently sparse and long, and (ii) there exist efficient algorithms that recover some shift truncation of the ground truth under the same conditions. In addition, the geometric characterization of the local solution as well as the proposed algorithm naturally extend to more complicated sparse blind deconvolution problems, including image deblurring, convolutional dictionary learning. We next consider the problem of modeling physical nuisances across a collection of images, in the context of illumination-invariant object detection and recognition. Illumination variation remains a central challenge in object detection and recognition. Existing analyses of illumination variation typically pertain to convex, Lambertian objects, and guarantee quality of approximation in an average case sense. We show that it is possible to build vertex-description convex cone models with worst-case performance guarantees, for nonconvex Lambertian objects. Namely, a natural detection test based on the angle to the constructed cone guarantees to accept any image which is sufficiently well approximated with an image of the object under some admissible lighting condition, and guarantees to reject any image that does not have a sufficiently approximation. The cone models are generated by sampling point illuminations with sufficient density, which follows from a new perturbation bound for point images in the Lambertian model. As the number of point images required for guaranteed detection may be large, we introduce a new formulation for cone preserving dimensionality reduction, which leverages tools from sparse and low-rank decomposition to reduce the complexity, while controlling the approximation error with respect to the original cone. Preliminary numerical experiments suggest that this approach can significantly reduce the complexity of the resulting model

Semidefinite Programming. methods and algorithms for energy management

La présente thèse a pour objet d explorer les potentialités d une méthode prometteuse de l optimisation conique, la programmation semi-définie positive (SDP), pour les problèmes de management d énergie, à savoir relatifs à la satisfaction des équilibres offre-demande électrique et gazier.Nos travaux se déclinent selon deux axes. Tout d abord nous nous intéressons à l utilisation de la SDP pour produire des relaxations de problèmes combinatoires et quadratiques. Si une relaxation SDP dite standard peut être élaborée très simplement, il est généralement souhaitable de la renforcer par des coupes, pouvant être déterminées par l'étude de la structure du problème ou à l'aide de méthodes plus systématiques. Nous mettons en œuvre ces deux approches sur différentes modélisations du problème de planification des arrêts nucléaires, réputé pour sa difficulté combinatoire. Nous terminons sur ce sujet par une expérimentation de la hiérarchie de Lasserre, donnant lieu à une suite de SDP dont la valeur optimale tend vers la solution du problème initial.Le second axe de la thèse porte sur l'application de la SDP à la prise en compte de l'incertitude. Nous mettons en œuvre une approche originale dénommée optimisation distributionnellement robuste , pouvant être vue comme un compromis entre optimisation stochastique et optimisation robuste et menant à des approximations sous forme de SDP. Nous nous appliquons à estimer l'apport de cette approche sur un problème d'équilibre offre-demande avec incertitude. Puis, nous présentons une relaxation SDP pour les problèmes MISOCP. Cette relaxation se révèle être de très bonne qualité, tout en ne nécessitant qu un temps de calcul raisonnable. La SDP se confirme donc être une méthode d optimisation prometteuse qui offre de nombreuses opportunités d'innovation en management d énergie.The present thesis aims at exploring the potentialities of a powerful optimization technique, namely Semidefinite Programming, for addressing some difficult problems of energy management. We pursue two main objectives. The first one consists of using SDP to provide tight relaxations of combinatorial and quadratic problems. A first relaxation, called standard can be derived in a generic way but it is generally desirable to reinforce them, by means of tailor-made tools or in a systematic fashion. These two approaches are implemented on different models of the Nuclear Outages Scheduling Problem, a famous combinatorial problem. We conclude this topic by experimenting the Lasserre's hierarchy on this problem, leading to a sequence of semidefinite relaxations whose optimal values tends to the optimal value of the initial problem.The second objective deals with the use of SDP for the treatment of uncertainty. We investigate an original approach called distributionnally robust optimization , that can be seen as a compromise between stochastic and robust optimization and admits approximations under the form of a SDP. We compare the benefits of this method w.r.t classical approaches on a demand/supply equilibrium problem. Finally, we propose a scheme for deriving SDP relaxations of MISOCP and we report promising computational results indicating that the semidefinite relaxation improves significantly the continuous relaxation, while requiring a reasonable computational effort.SDP therefore proves to be a promising optimization method that offers great opportunities for innovation in energy management.PARIS11-SCD-Bib. électronique (914719901) / SudocSudocFranceF

Semidefinite Relaxations for Lebesgue and Gaussian Measures of Unions of Basic Semialgebraic Sets

International audienceGiven a finite Borel measure µ on R n and basic semi-algebraic sets Ω_i ⊂ R n , i = 1,. .. , p, we provide a systematic numerical scheme to approximate as closely as desired µ(\cup_i Ω_i), when all moments of µ are available (and finite). More precisely , we provide a hierarchy of semidefinite programs whose associated sequence of optimal values is monotone and converges to the desired value from above. The same methodology applied to the complement R n \ (\cup_i Ω_i) provides a monotone sequence that converges to the desired value from below. When µ is the Lebesgue measure we assume that Ω := \cup_i Ω_i is compact and contained in a known box B and in this case the complement is taken to be B \ Ω. In fact, not only µ(Ω) but also every finite vector of moments of µ_Ω (the restriction of µ on Ω) can be approximated as closely as desired, and so permits to approximate the integral on Ω of any given polynomial