Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

New Dependencies of Hierarchies in Polynomial Optimization

We compare four key hierarchies for solving Constrained Polynomial Optimization Problems (CPOP): Sum of Squares (SOS), Sum of Diagonally Dominant Polynomials (SDSOS), Sum of Nonnegative Circuits (SONC), and the Sherali Adams (SA) hierarchies. We prove a collection of dependencies among these hierarchies both for general CPOPs and for optimization problems on the Boolean hypercube. Key results include for the general case that the SONC and SOS hierarchy are polynomially incomparable, while SDSOS is contained in SONC. A direct consequence is the non-existence of a Putinar-like Positivstellensatz for SDSOS. On the Boolean hypercube, we show as a main result that Schm\"udgen-like versions of the hierarchies SDSOS*, SONC*, and SA* are polynomially equivalent. Moreover, we show that SA* is contained in any Schm\"udgen-like hierarchy that provides a O(n) degree bound.Comment: 26 pages, 4 figure

Conic Programming Approaches for Polynomial Optimization: Theory and Applications

Historically, polynomials are among the most popular class of functions used for empirical modeling in science and engineering. Polynomials are easy to evaluate, appear naturally in many physical (real-world) systems, and can be used to accurately approximate any smooth function. It is not surprising then, that the task of solving polynomial optimization problems; that is, problems where both the objective function and constraints are multivariate polynomials, is ubiquitous and of enormous interest in these fields. Clearly, polynomial op- timization problems encompass a very general class of non-convex optimization problems, including key combinatorial optimization problems.The focus of the first three chapters of this document is to address the solution of polynomial optimization problems in theory and in practice, using a conic optimization approach. Convex optimization has been well studied to solve quadratic constrained quadratic problems. In the first part, convex relaxations for general polynomial optimization problems are discussed. Instead of using the matrix space to study quadratic programs, we study the convex relaxations for POPs through a lifted tensor space, more specifically, using the completely positive tensor cone and the completely positive semidefinite tensor cone. We show that tensor relaxations theoretically yield no-worse global bounds for a class of polynomial optimization problems than relaxation for a QCQP reformulation of the POPs. We also propose an approximation strategy for tensor cones and show empirically the advantage of the tensor relaxation.In the second part, we propose an alternative SDP and SOCP hierarchy to obtain global bounds for general polynomial optimization problems. Comparing with other existing SDP and SOCP hierarchies that uses higher degree sum of square (SOS) polynomials and scaled diagonally sum of square polynomials (SDSOS) when the hierarchy level increases, these proposed hierarchies, using fixed degree SOS and SDSOS polynomials but more of these polynomials, perform numerically better. Numerical results show that the hierarchies we proposed have better performance in terms of tightness of the bound and solution time compared with other hierarchies in the literature.The third chapter deals with Alternating Current Optimal Power Flow problem via a polynomial optimization approach. The Alternating Current Optimal Power Flow (ACOPF) problem is a challenging non-convex optimization problem in power systems. Prior research mainly focuses on using SDP relaxations and SDP-based hierarchies to address the solution of ACOPF problem. In this Chapter, we apply existing SOCP hierarchies to this problem and explore the structure of the network to propose simplified hierarchies for ACOPF problems. Compared with SDP approaches, SOCP approaches are easier to solve and can be used to approximate large scale ACOPF problems.The last chapter also relates to the use of conic optimization techniques, but in this case to pricing in markets with non-convexities. Indeed, it is an application of conic optimization approach to solve a pricing problem in energy systems. Prior research in energy market pricing mainly focus on linear costs in the objective function. Due to the penetration of renewable energies into the current electricity grid, it is important to consider quadratic costs in the objective function, which reflects the ramping costs for traditional generators. This study address the issue how to find the market clearing prices when considering quadratic costs in the objective function

Nonlinear Integer Programming

Research efforts of the past fifty years have led to a development of linear integer programming as a mature discipline of mathematical optimization. Such a level of maturity has not been reached when one considers nonlinear systems subject to integrality requirements for the variables. This chapter is dedicated to this topic. The primary goal is a study of a simple version of general nonlinear integer problems, where all constraints are still linear. Our focus is on the computational complexity of the problem, which varies significantly with the type of nonlinear objective function in combination with the underlying combinatorial structure. Numerous boundary cases of complexity emerge, which sometimes surprisingly lead even to polynomial time algorithms. We also cover recent successful approaches for more general classes of problems. Though no positive theoretical efficiency results are available, nor are they likely to ever be available, these seem to be the currently most successful and interesting approaches for solving practical problems. It is our belief that the study of algorithms motivated by theoretical considerations and those motivated by our desire to solve practical instances should and do inform one another. So it is with this viewpoint that we present the subject, and it is in this direction that we hope to spark further research.Comment: 57 pages. To appear in: M. J\"unger, T. Liebling, D. Naddef, G. Nemhauser, W. Pulleyblank, G. Reinelt, G. Rinaldi, and L. Wolsey (eds.), 50 Years of Integer Programming 1958--2008: The Early Years and State-of-the-Art Surveys, Springer-Verlag, 2009, ISBN 354068274

Copositive certificates of non-negativity for polynomials on semialgebraic sets

A certificate of non-negativity is a way to write a given function so that its non-negativity becomes evident. Certificates of non-negativity are fundamental tools in optimization, and they underlie powerful algorithmic techniques for various types of optimization problems. We propose certificates of non-negativity of polynomials based on copositive polynomials. The certificates we obtain are valid for generic semialgebraic sets and have a fixed small degree, while commonly used sums-of-squares (SOS) certificates are guaranteed to be valid only for compact semialgebraic sets and could have large degree. Optimization over the cone of copositive polynomials is not tractable, but this cone has been well studied. The main benefit of our copositive certificates of non-negativity is their ability to translate results known exclusively for copositive polynomials to more general semialgebraic sets. In particular, we show how to use copositive polynomials to construct structured (e.g., sparse) certificates of non-negativity, even for unstructured semialgebraic sets. Last but not least, copositive certificates can be used to obtain not only hierarchies of tractable lower bounds, but also hierarchies of tractable upper bounds for polynomial optimization problems.Comment: 27 pages, 1 figur