2,741 research outputs found
Tightness of the maximum likelihood semidefinite relaxation for angular synchronization
Maximum likelihood estimation problems are, in general, intractable
optimization problems. As a result, it is common to approximate the maximum
likelihood estimator (MLE) using convex relaxations. In some cases, the
relaxation is tight: it recovers the true MLE. Most tightness proofs only apply
to situations where the MLE exactly recovers a planted solution (known to the
analyst). It is then sufficient to establish that the optimality conditions
hold at the planted signal. In this paper, we study an estimation problem
(angular synchronization) for which the MLE is not a simple function of the
planted solution, yet for which the convex relaxation is tight. To establish
tightness in this context, the proof is less direct because the point at which
to verify optimality conditions is not known explicitly.
Angular synchronization consists in estimating a collection of phases,
given noisy measurements of the pairwise relative phases. The MLE for angular
synchronization is the solution of a (hard) non-bipartite Grothendieck problem
over the complex numbers. We consider a stochastic model for the data: a
planted signal (that is, a ground truth set of phases) is corrupted with
non-adversarial random noise. Even though the MLE does not coincide with the
planted signal, we show that the classical semidefinite relaxation for it is
tight, with high probability. This holds even for high levels of noise.Comment: 2 figure
Natural evolution strategies and variational Monte Carlo
A notion of quantum natural evolution strategies is introduced, which
provides a geometric synthesis of a number of known quantum/classical
algorithms for performing classical black-box optimization. Recent work of
Gomes et al. [2019] on heuristic combinatorial optimization using neural
quantum states is pedagogically reviewed in this context, emphasizing the
connection with natural evolution strategies. The algorithmic framework is
illustrated for approximate combinatorial optimization problems, and a
systematic strategy is found for improving the approximation ratios. In
particular it is found that natural evolution strategies can achieve
approximation ratios competitive with widely used heuristic algorithms for
Max-Cut, at the expense of increased computation time
Low-rank semidefinite programming for the MAX2SAT problem
This paper proposes a new algorithm for solving MAX2SAT problems based on
combining search methods with semidefinite programming approaches. Semidefinite
programming techniques are well-known as a theoretical tool for approximating
maximum satisfiability problems, but their application has traditionally been
very limited by their speed and randomized nature. Our approach overcomes this
difficult by using a recent approach to low-rank semidefinite programming,
specialized to work in an incremental fashion suitable for use in an exact
search algorithm. The method can be used both within complete or incomplete
solver, and we demonstrate on a variety of problems from recent competitions.
Our experiments show that the approach is faster (sometimes by orders of
magnitude) than existing state-of-the-art complete and incomplete solvers,
representing a substantial advance in search methods specialized for MAX2SAT
problems.Comment: Accepted at AAAI'19. The code can be found at
https://github.com/locuslab/mixsa
Approximate Graph Coloring by Semidefinite Programming
We consider the problem of coloring k-colorable graphs with the fewest
possible colors. We present a randomized polynomial time algorithm that colors
a 3-colorable graph on vertices with min O(Delta^{1/3} log^{1/2} Delta log
n), O(n^{1/4} log^{1/2} n) colors where Delta is the maximum degree of any
vertex. Besides giving the best known approximation ratio in terms of n, this
marks the first non-trivial approximation result as a function of the maximum
degree Delta. This result can be generalized to k-colorable graphs to obtain a
coloring using min O(Delta^{1-2/k} log^{1/2} Delta log n), O(n^{1-3/(k+1)}
log^{1/2} n) colors. Our results are inspired by the recent work of Goemans and
Williamson who used an algorithm for semidefinite optimization problems, which
generalize linear programs, to obtain improved approximations for the MAX CUT
and MAX 2-SAT problems. An intriguing outcome of our work is a duality
relationship established between the value of the optimum solution to our
semidefinite program and the Lovasz theta-function. We show lower bounds on the
gap between the optimum solution of our semidefinite program and the actual
chromatic number; by duality this also demonstrates interesting new facts about
the theta-function
Efficient Semidefinite Branch-and-Cut for MAP-MRF Inference
We propose a Branch-and-Cut (B&C) method for solving general MAP-MRF
inference problems. The core of our method is a very efficient bounding
procedure, which combines scalable semidefinite programming (SDP) and a
cutting-plane method for seeking violated constraints. In order to further
speed up the computation, several strategies have been exploited, including
model reduction, warm start and removal of inactive constraints.
We analyze the performance of the proposed method under different settings,
and demonstrate that our method either outperforms or performs on par with
state-of-the-art approaches. Especially when the connectivities are dense or
when the relative magnitudes of the unary costs are low, we achieve the best
reported results. Experiments show that the proposed algorithm achieves better
approximation than the state-of-the-art methods within a variety of time
budgets on challenging non-submodular MAP-MRF inference problems.Comment: 21 page
Multireference Alignment using Semidefinite Programming
The multireference alignment problem consists of estimating a signal from
multiple noisy shifted observations. Inspired by existing Unique-Games
approximation algorithms, we provide a semidefinite program (SDP) based
relaxation which approximates the maximum likelihood estimator (MLE) for the
multireference alignment problem. Although we show that the MLE problem is
Unique-Games hard to approximate within any constant, we observe that our
poly-time approximation algorithm for the MLE appears to perform quite well in
typical instances, outperforming existing methods. In an attempt to explain
this behavior we provide stability guarantees for our SDP under a random noise
model on the observations. This case is more challenging to analyze than
traditional semi-random instances of Unique-Games: the noise model is on
vertices of a graph and translates into dependent noise on the edges.
Interestingly, we show that if certain positivity constraints in the SDP are
dropped, its solution becomes equivalent to performing phase correlation, a
popular method used for pairwise alignment in imaging applications. Finally, we
show how symmetry reduction techniques from matrix representation theory can
simplify the analysis and computation of the SDP, greatly decreasing its
computational cost
A note on Probably Certifiably Correct algorithms
Many optimization problems of interest are known to be intractable, and while
there are often heuristics that are known to work on typical instances, it is
usually not easy to determine a posteriori whether the optimal solution was
found. In this short note, we discuss algorithms that not only solve the
problem on typical instances, but also provide a posteriori certificates of
optimality, probably certifiably correct (PCC) algorithms. As an illustrative
example, we present a fast PCC algorithm for minimum bisection under the
stochastic block model and briefly discuss other examples
Approximating the Little Grothendieck Problem over the Orthogonal and Unitary Groups
The little Grothendieck problem consists of maximizing
over binary variables , where C is a
positive semidefinite matrix. In this paper we focus on a natural
generalization of this problem, the little Grothendieck problem over the
orthogonal group. Given C a dn x dn positive semidefinite matrix, the objective
is to maximize restricting to take
values in the group of orthogonal matrices, where denotes the (ij)-th
d x d block of C. We propose an approximation algorithm, which we refer to as
Orthogonal-Cut, to solve this problem and show a constant approximation ratio.
Our method is based on semidefinite programming. For a given , we show
a constant approximation ratio of , where is
the expected average singular value of a d x d matrix with random Gaussian
i.i.d. entries. For d=1 we recover the known
approximation guarantee for the classical little Grothendieck problem. Our
algorithm and analysis naturally extends to the complex valued case also
providing a constant approximation ratio for the analogous problem over the
Unitary Group.
Orthogonal-Cut also serves as an approximation algorithm for several
applications, including the Procrustes problem where it improves over the best
previously known approximation ratio of~. The little
Grothendieck problem falls under the class of problems approximated by a recent
algorithm proposed in the context of the non-commutative Grothendieck
inequality. Nonetheless, our approach is simpler and it provides a more
efficient algorithm with better approximation ratios and matching integrality
gaps.
Finally, we also provide an improved approximation algorithm for the more
general little Grothendieck problem over the orthogonal (or unitary) group with
rank constraints.Comment: Updates in version 2: extension to the complex valued (unitary group)
case, sharper lower bounds on the approximation ratios, matching integrality
gap, and a generalized rank constrained version of the problem. Updates in
version 3: Improvement on the expositio
A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints
We propose a new algorithm to solve optimization problems of the form for a smooth function under the constraints that is positive
semidefinite and the diagonal blocks of are small identity matrices. Such
problems often arise as the result of relaxing a rank constraint (lifting). In
particular, many estimation tasks involving phases, rotations, orthonormal
bases or permutations fit in this framework, and so do certain relaxations of
combinatorial problems such as Max-Cut. The proposed algorithm exploits the
facts that (1) such formulations admit low-rank solutions, and (2) their
rank-restricted versions are smooth optimization problems on a Riemannian
manifold. Combining insights from both the Riemannian and the convex geometries
of the problem, we characterize when second-order critical points of the smooth
problem reveal KKT points of the semidefinite problem. We compare against state
of the art, mature software and find that, on certain interesting problem
instances, what we call the staircase method is orders of magnitude faster, is
more accurate and scales better. Code is available.Comment: 37 pages, 3 figure
- âŠ