715 research outputs found
Lower bounds on the size of semidefinite programming relaxations
We introduce a method for proving lower bounds on the efficacy of
semidefinite programming (SDP) relaxations for combinatorial problems. In
particular, we show that the cut, TSP, and stable set polytopes on -vertex
graphs are not the linear image of the feasible region of any SDP (i.e., any
spectrahedron) of dimension less than , for some constant .
This result yields the first super-polynomial lower bounds on the semidefinite
extension complexity of any explicit family of polytopes.
Our results follow from a general technique for proving lower bounds on the
positive semidefinite rank of a matrix. To this end, we establish a close
connection between arbitrary SDPs and those arising from the sum-of-squares SDP
hierarchy. For approximating maximum constraint satisfaction problems, we prove
that SDPs of polynomial-size are equivalent in power to those arising from
degree- sum-of-squares relaxations. This result implies, for instance,
that no family of polynomial-size SDP relaxations can achieve better than a
7/8-approximation for MAX-3-SAT
Lift & Project Systems Performing on the Partial-Vertex-Cover Polytope
We study integrality gap (IG) lower bounds on strong LP and SDP relaxations
derived by the Sherali-Adams (SA), Lovasz-Schrijver-SDP (LS+), and
Sherali-Adams-SDP (SA+) lift-and-project (L&P) systems for the
t-Partial-Vertex-Cover (t-PVC) problem, a variation of the classic Vertex-Cover
problem in which only t edges need to be covered. t-PVC admits a
2-approximation using various algorithmic techniques, all relying on a natural
LP relaxation. Starting from this LP relaxation, our main results assert that
for every epsilon > 0, level-Theta(n) LPs or SDPs derived by all known L&P
systems that have been used for positive algorithmic results (but the Lasserre
hierarchy) have IGs at least (1-epsilon)n/t, where n is the number of vertices
of the input graph. Our lower bounds are nearly tight.
Our results show that restricted yet powerful models of computation derived
by many L&P systems fail to witness c-approximate solutions to t-PVC for any
constant c, and for t = O(n). This is one of the very few known examples of an
intractable combinatorial optimization problem for which LP-based algorithms
induce a constant approximation ratio, still lift-and-project LP and SDP
tightenings of the same LP have unbounded IGs.
We also show that the SDP that has given the best algorithm known for t-PVC
has integrality gap n/t on instances that can be solved by the level-1 LP
relaxation derived by the LS system. This constitutes another rare phenomenon
where (even in specific instances) a static LP outperforms an SDP that has been
used for the best approximation guarantee for the problem at hand. Finally, one
of our main contributions is that we make explicit of a new and simple
methodology of constructing solutions to LP relaxations that almost trivially
satisfy constraints derived by all SDP L&P systems known to be useful for
algorithmic positive results (except the La system).Comment: 26 page
Conic Optimization Theory: Convexification Techniques and Numerical Algorithms
Optimization is at the core of control theory and appears in several areas of
this field, such as optimal control, distributed control, system
identification, robust control, state estimation, model predictive control and
dynamic programming. The recent advances in various topics of modern
optimization have also been revamping the area of machine learning. Motivated
by the crucial role of optimization theory in the design, analysis, control and
operation of real-world systems, this tutorial paper offers a detailed overview
of some major advances in this area, namely conic optimization and its emerging
applications. First, we discuss the importance of conic optimization in
different areas. Then, we explain seminal results on the design of hierarchies
of convex relaxations for a wide range of nonconvex problems. Finally, we study
different numerical algorithms for large-scale conic optimization problems.Comment: 18 page
Relax, no need to round: integrality of clustering formulations
We study exact recovery conditions for convex relaxations of point cloud
clustering problems, focusing on two of the most common optimization problems
for unsupervised clustering: -means and -median clustering. Motivations
for focusing on convex relaxations are: (a) they come with a certificate of
optimality, and (b) they are generic tools which are relatively parameter-free,
not tailored to specific assumptions over the input. More precisely, we
consider the distributional setting where there are clusters in
and data from each cluster consists of points sampled from a
symmetric distribution within a ball of unit radius. We ask: what is the
minimal separation distance between cluster centers needed for convex
relaxations to exactly recover these clusters as the optimal integral
solution? For the -median linear programming relaxation we show a tight
bound: exact recovery is obtained given arbitrarily small pairwise separation
between the balls. In other words, the pairwise center
separation is . Under the same distributional model, the
-means LP relaxation fails to recover such clusters at separation as large
as . Yet, if we enforce PSD constraints on the -means LP, we get
exact cluster recovery at center separation .
In contrast, common heuristics such as Lloyd's algorithm (a.k.a. the -means
algorithm) can fail to recover clusters in this setting; even with arbitrarily
large cluster separation, k-means++ with overseeding by any constant factor
fails with high probability at exact cluster recovery. To complement the
theoretical analysis, we provide an experimental study of the recovery
guarantees for these various methods, and discuss several open problems which
these experiments suggest.Comment: 30 pages, ITCS 201
- …