1,130 research outputs found
Improving Efficiency and Scalability of Sum of Squares Optimization: Recent Advances and Limitations
It is well-known that any sum of squares (SOS) program can be cast as a
semidefinite program (SDP) of a particular structure and that therein lies the
computational bottleneck for SOS programs, as the SDPs generated by this
procedure are large and costly to solve when the polynomials involved in the
SOS programs have a large number of variables and degree. In this paper, we
review SOS optimization techniques and present two new methods for improving
their computational efficiency. The first method leverages the sparsity of the
underlying SDP to obtain computational speed-ups. Further improvements can be
obtained if the coefficients of the polynomials that describe the problem have
a particular sparsity pattern, called chordal sparsity. The second method
bypasses semidefinite programming altogether and relies instead on solving a
sequence of more tractable convex programs, namely linear and second order cone
programs. This opens up the question as to how well one can approximate the
cone of SOS polynomials by second order representable cones. In the last part
of the paper, we present some recent negative results related to this question.Comment: Tutorial for CDC 201
Approximation Limits of Linear Programs (Beyond Hierarchies)
We develop a framework for approximation limits of polynomial-size linear
programs from lower bounds on the nonnegative ranks of suitably defined
matrices. This framework yields unconditional impossibility results that are
applicable to any linear program as opposed to only programs generated by
hierarchies. Using our framework, we prove that O(n^{1/2-eps})-approximations
for CLIQUE require linear programs of size 2^{n^\Omega(eps)}. (This lower bound
applies to linear programs using a certain encoding of CLIQUE as a linear
optimization problem.) Moreover, we establish a similar result for
approximations of semidefinite programs by linear programs. Our main ingredient
is a quantitative improvement of Razborov's rectangle corruption lemma for the
high error regime, which gives strong lower bounds on the nonnegative rank of
certain perturbations of the unique disjointness matrix.Comment: 23 pages, 2 figure
Sparse sum-of-squares (SOS) optimization: A bridge between DSOS/SDSOS and SOS optimization for sparse polynomials
Optimization over non-negative polynomials is fundamental for nonlinear
systems analysis and control. We investigate the relation between three
tractable relaxations for optimizing over sparse non-negative polynomials:
sparse sum-of-squares (SSOS) optimization, diagonally dominant sum-of-squares
(DSOS) optimization, and scaled diagonally dominant sum-of-squares (SDSOS)
optimization. We prove that the set of SSOS polynomials, an inner approximation
of the cone of SOS polynomials, strictly contains the spaces of sparse
DSOS/SDSOS polynomials. When applicable, therefore, SSOS optimization is less
conservative than its DSOS/SDSOS counterparts. Numerical results for
large-scale sparse polynomial optimization problems demonstrate this fact, and
also that SSOS optimization can be faster than DSOS/SDSOS methods despite
requiring the solution of semidefinite programs instead of less expensive
linear/second-order cone programs.Comment: 9 pages, 3 figure
Positive Semidefinite Metric Learning Using Boosting-like Algorithms
The success of many machine learning and pattern recognition methods relies
heavily upon the identification of an appropriate distance metric on the input
data. It is often beneficial to learn such a metric from the input training
data, instead of using a default one such as the Euclidean distance. In this
work, we propose a boosting-based technique, termed BoostMetric, for learning a
quadratic Mahalanobis distance metric. Learning a valid Mahalanobis distance
metric requires enforcing the constraint that the matrix parameter to the
metric remains positive definite. Semidefinite programming is often used to
enforce this constraint, but does not scale well and easy to implement.
BoostMetric is instead based on the observation that any positive semidefinite
matrix can be decomposed into a linear combination of trace-one rank-one
matrices. BoostMetric thus uses rank-one positive semidefinite matrices as weak
learners within an efficient and scalable boosting-based learning process. The
resulting methods are easy to implement, efficient, and can accommodate various
types of constraints. We extend traditional boosting algorithms in that its
weak learner is a positive semidefinite matrix with trace and rank being one
rather than a classifier or regressor. Experiments on various datasets
demonstrate that the proposed algorithms compare favorably to those
state-of-the-art methods in terms of classification accuracy and running time.Comment: 30 pages, appearing in Journal of Machine Learning Researc
Decomposed Structured Subsets for Semidefinite and Sum-of-Squares Optimization
Semidefinite programs (SDPs) are standard convex problems that are frequently
found in control and optimization applications. Interior-point methods can
solve SDPs in polynomial time up to arbitrary accuracy, but scale poorly as the
size of matrix variables and the number of constraints increases. To improve
scalability, SDPs can be approximated with lower and upper bounds through the
use of structured subsets (e.g., diagonally-dominant and scaled-diagonally
dominant matrices). Meanwhile, any underlying sparsity or symmetry structure
may be leveraged to form an equivalent SDP with smaller positive semidefinite
constraints. In this paper, we present a notion of decomposed structured
subsets}to approximate an SDP with structured subsets after an equivalent
conversion. The lower/upper bounds found by approximation after conversion
become tighter than the bounds obtained by approximating the original SDP
directly. We apply decomposed structured subsets to semidefinite and
sum-of-squares optimization problems with examples of H-infinity norm
estimation and constrained polynomial optimization. An existing basis pursuit
method is adapted into this framework to iteratively refine bounds.Comment: 23 pages, 10 figures, 9 table
- …