196 research outputs found
Maximum Persistency via Iterative Relaxed Inference with Graphical Models
We consider the NP-hard problem of MAP-inference for undirected discrete
graphical models. We propose a polynomial time and practically efficient
algorithm for finding a part of its optimal solution. Specifically, our
algorithm marks some labels of the considered graphical model either as (i)
optimal, meaning that they belong to all optimal solutions of the inference
problem; (ii) non-optimal if they provably do not belong to any solution. With
access to an exact solver of a linear programming relaxation to the
MAP-inference problem, our algorithm marks the maximal possible (in a specified
sense) number of labels. We also present a version of the algorithm, which has
access to a suboptimal dual solver only and still can ensure the
(non-)optimality for the marked labels, although the overall number of the
marked labels may decrease. We propose an efficient implementation, which runs
in time comparable to a single run of a suboptimal dual solver. Our method is
well-scalable and shows state-of-the-art results on computational benchmarks
from machine learning and computer vision.Comment: Reworked version, submitted to PAM
Maximum Persistency in Energy Minimization
We consider discrete pairwise energy minimization problem (weighted
constraint satisfaction, max-sum labeling) and methods that identify a globally
optimal partial assignment of variables. When finding a complete optimal
assignment is intractable, determining optimal values for a part of variables
is an interesting possibility. Existing methods are based on different
sufficient conditions. We propose a new sufficient condition for partial
optimality which is: (1) verifiable in polynomial time (2) invariant to
reparametrization of the problem and permutation of labels and (3) includes
many existing sufficient conditions as special cases. We pose the problem of
finding the maximum optimal partial assignment identifiable by the new
sufficient condition. A polynomial method is proposed which is guaranteed to
assign same or larger part of variables than several existing approaches. The
core of the method is a specially constructed linear program that identifies
persistent assignments in an arbitrary multi-label setting.Comment: Extended technical report for the CVPR 2014 paper. Update: correction
to the proof of characterization theore
Combinatorial persistency criteria for multicut and max-cut
In combinatorial optimization, partial variable assignments are called
persistent if they agree with some optimal solution. We propose persistency
criteria for the multicut and max-cut problem as well as fast combinatorial
routines to verify them. The criteria that we derive are based on mappings that
improve feasible multicuts, respectively cuts. Our elementary criteria can be
checked enumeratively. The more advanced ones rely on fast algorithms for upper
and lower bounds for the respective cut problems and max-flow techniques for
auxiliary min-cut problems. Our methods can be used as a preprocessing
technique for reducing problem sizes or for computing partial optimality
guarantees for solutions output by heuristic solvers. We show the efficacy of
our methods on instances of both problems from computer vision, biomedical
image analysis and statistical physics
Efficient Semidefinite Branch-and-Cut for MAP-MRF Inference
We propose a Branch-and-Cut (B&C) method for solving general MAP-MRF
inference problems. The core of our method is a very efficient bounding
procedure, which combines scalable semidefinite programming (SDP) and a
cutting-plane method for seeking violated constraints. In order to further
speed up the computation, several strategies have been exploited, including
model reduction, warm start and removal of inactive constraints.
We analyze the performance of the proposed method under different settings,
and demonstrate that our method either outperforms or performs on par with
state-of-the-art approaches. Especially when the connectivities are dense or
when the relative magnitudes of the unary costs are low, we achieve the best
reported results. Experiments show that the proposed algorithm achieves better
approximation than the state-of-the-art methods within a variety of time
budgets on challenging non-submodular MAP-MRF inference problems.Comment: 21 page
A discriminative view of MRF pre-processing algorithms
While Markov Random Fields (MRFs) are widely used in computer vision, they
present a quite challenging inference problem. MRF inference can be accelerated
by pre-processing techniques like Dead End Elimination (DEE) or QPBO-based
approaches which compute the optimal labeling of a subset of variables. These
techniques are guaranteed to never wrongly label a variable but they often
leave a large number of variables unlabeled. We address this shortcoming by
interpreting pre-processing as a classification problem, which allows us to
trade off false positives (i.e., giving a variable an incorrect label) versus
false negatives (i.e., failing to label a variable). We describe an efficient
discriminative rule that finds optimal solutions for a subset of variables. Our
technique provides both per-instance and worst-case guarantees concerning the
quality of the solution. Empirical studies were conducted over several
benchmark datasets. We obtain a speedup factor of 2 to 12 over expansion moves
without preprocessing, and on difficult non-submodular energy functions produce
slightly lower energy.Comment: ICCV 201
Complexity of Discrete Energy Minimization Problems
Discrete energy minimization is widely-used in computer vision and machine
learning for problems such as MAP inference in graphical models. The problem,
in general, is notoriously intractable, and finding the global optimal solution
is known to be NP-hard. However, is it possible to approximate this problem
with a reasonable ratio bound on the solution quality in polynomial time? We
show in this paper that the answer is no. Specifically, we show that general
energy minimization, even in the 2-label pairwise case, and planar energy
minimization with three or more labels are exp-APX-complete. This finding rules
out the existence of any approximation algorithm with a sub-exponential
approximation ratio in the input size for these two problems, including
constant factor approximations. Moreover, we collect and review the
computational complexity of several subclass problems and arrange them on a
complexity scale consisting of three major complexity classes -- PO, APX, and
exp-APX, corresponding to problems that are solvable, approximable, and
inapproximable in polynomial time. Problems in the first two complexity classes
can serve as alternative tractable formulations to the inapproximable ones.
This paper can help vision researchers to select an appropriate model for an
application or guide them in designing new algorithms.Comment: ECCV'16 accepte
Structured Prediction Problem Archive
Structured prediction problems are one of the fundamental tools in machinelearning. In order to facilitate algorithm development for their numericalsolution, we collect in one place a large number of datasets in easy to readformats for a diverse set of problem classes. We provide archival links todatasets, description of the considered problems and problem formats, and ashort summary of problem characteristics including size, number of instancesetc. For reference we also give a non-exhaustive selection of algorithmsproposed in the literature for their solution. We hope that this centralrepository will make benchmarking and comparison to established works easier.We welcome submission of interesting new datasets and algorithms for inclusionin our archive.<br
GraphLab: A New Framework for Parallel Machine Learning
Designing and implementing efficient, provably correct parallel machine
learning (ML) algorithms is challenging. Existing high-level parallel
abstractions like MapReduce are insufficiently expressive while low-level tools
like MPI and Pthreads leave ML experts repeatedly solving the same design
challenges. By targeting common patterns in ML, we developed GraphLab, which
improves upon abstractions like MapReduce by compactly expressing asynchronous
iterative algorithms with sparse computational dependencies while ensuring data
consistency and achieving a high degree of parallel performance. We demonstrate
the expressiveness of the GraphLab framework by designing and implementing
parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and
Compressed Sensing. We show that using GraphLab we can achieve excellent
parallel performance on large scale real-world problems
- …