26,855 research outputs found
Complexity of Discrete Energy Minimization Problems
Discrete energy minimization is widely-used in computer vision and machine
learning for problems such as MAP inference in graphical models. The problem,
in general, is notoriously intractable, and finding the global optimal solution
is known to be NP-hard. However, is it possible to approximate this problem
with a reasonable ratio bound on the solution quality in polynomial time? We
show in this paper that the answer is no. Specifically, we show that general
energy minimization, even in the 2-label pairwise case, and planar energy
minimization with three or more labels are exp-APX-complete. This finding rules
out the existence of any approximation algorithm with a sub-exponential
approximation ratio in the input size for these two problems, including
constant factor approximations. Moreover, we collect and review the
computational complexity of several subclass problems and arrange them on a
complexity scale consisting of three major complexity classes -- PO, APX, and
exp-APX, corresponding to problems that are solvable, approximable, and
inapproximable in polynomial time. Problems in the first two complexity classes
can serve as alternative tractable formulations to the inapproximable ones.
This paper can help vision researchers to select an appropriate model for an
application or guide them in designing new algorithms.Comment: ECCV'16 accepte
Incidence Geometries and the Pass Complexity of Semi-Streaming Set Cover
Set cover, over a universe of size , may be modelled as a data-streaming
problem, where the sets that comprise the instance are to be read one by
one. A semi-streaming algorithm is allowed only space to process this stream. For each , we give a very
simple deterministic algorithm that makes passes over the input stream and
returns an appropriately certified -approximation to the
optimum set cover. More importantly, we proceed to show that this approximation
factor is essentially tight, by showing that a factor better than
is unachievable for a -pass semi-streaming
algorithm, even allowing randomisation. In particular, this implies that
achieving a -approximation requires
passes, which is tight up to the factor. These results extend to a
relaxation of the set cover problem where we are allowed to leave an
fraction of the universe uncovered: the tight bounds on the best
approximation factor achievable in passes turn out to be
. Our lower bounds are based
on a construction of a family of high-rank incidence geometries, which may be
thought of as vast generalisations of affine planes. This construction, based
on algebraic techniques, appears flexible enough to find other applications and
is therefore interesting in its own right.Comment: 20 page
Approximations of Semicontinuous Functions with Applications to Stochastic Optimization and Statistical Estimation
Upper semicontinuous (usc) functions arise in the analysis of maximization
problems, distributionally robust optimization, and function identification,
which includes many problems of nonparametric statistics. We establish that
every usc function is the limit of a hypo-converging sequence of piecewise
affine functions of the difference-of-max type and illustrate resulting
algorithmic possibilities in the context of approximate solution of
infinite-dimensional optimization problems. In an effort to quantify the ease
with which classes of usc functions can be approximated by finite collections,
we provide upper and lower bounds on covering numbers for bounded sets of usc
functions under the Attouch-Wets distance. The result is applied in the context
of stochastic optimization problems defined over spaces of usc functions. We
establish confidence regions for optimal solutions based on sample average
approximations and examine the accompanying rates of convergence. Examples from
nonparametric statistics illustrate the results
Approximate Graph Coloring by Semidefinite Programming
We consider the problem of coloring k-colorable graphs with the fewest
possible colors. We present a randomized polynomial time algorithm that colors
a 3-colorable graph on vertices with min O(Delta^{1/3} log^{1/2} Delta log
n), O(n^{1/4} log^{1/2} n) colors where Delta is the maximum degree of any
vertex. Besides giving the best known approximation ratio in terms of n, this
marks the first non-trivial approximation result as a function of the maximum
degree Delta. This result can be generalized to k-colorable graphs to obtain a
coloring using min O(Delta^{1-2/k} log^{1/2} Delta log n), O(n^{1-3/(k+1)}
log^{1/2} n) colors. Our results are inspired by the recent work of Goemans and
Williamson who used an algorithm for semidefinite optimization problems, which
generalize linear programs, to obtain improved approximations for the MAX CUT
and MAX 2-SAT problems. An intriguing outcome of our work is a duality
relationship established between the value of the optimum solution to our
semidefinite program and the Lovasz theta-function. We show lower bounds on the
gap between the optimum solution of our semidefinite program and the actual
chromatic number; by duality this also demonstrates interesting new facts about
the theta-function
Automatic Variational Inference in Stan
Variational inference is a scalable technique for approximate Bayesian
inference. Deriving variational inference algorithms requires tedious
model-specific calculations; this makes it difficult to automate. We propose an
automatic variational inference algorithm, automatic differentiation
variational inference (ADVI). The user only provides a Bayesian model and a
dataset; nothing else. We make no conjugacy assumptions and support a broad
class of models. The algorithm automatically determines an appropriate
variational family and optimizes the variational objective. We implement ADVI
in Stan (code available now), a probabilistic programming framework. We compare
ADVI to MCMC sampling across hierarchical generalized linear models,
nonconjugate matrix factorization, and a mixture model. We train the mixture
model on a quarter million images. With ADVI we can use variational inference
on any model we write in Stan
- …