332 research outputs found
Structured learning of sum-of-submodular higher order energy functions
Submodular functions can be exactly minimized in polynomial time, and the
special case that graph cuts solve with max flow \cite{KZ:PAMI04} has had
significant impact in computer vision
\cite{BVZ:PAMI01,Kwatra:SIGGRAPH03,Rother:GrabCut04}. In this paper we address
the important class of sum-of-submodular (SoS) functions
\cite{Arora:ECCV12,Kolmogorov:DAM12}, which can be efficiently minimized via a
variant of max flow called submodular flow \cite{Edmonds:ADM77}. SoS functions
can naturally express higher order priors involving, e.g., local image patches;
however, it is difficult to fully exploit their expressive power because they
have so many parameters. Rather than trying to formulate existing higher order
priors as an SoS function, we take a discriminative learning approach,
effectively searching the space of SoS functions for a higher order prior that
performs well on our training set. We adopt a structural SVM approach
\cite{Joachims/etal/09a,Tsochantaridis/etal/04} and formulate the training
problem in terms of quadratic programming; as a result we can efficiently
search the space of SoS priors via an extended cutting-plane algorithm. We also
show how the state-of-the-art max flow method for vision problems
\cite{Goldberg:ESA11} can be modified to efficiently solve the submodular flow
problem. Experimental comparisons are made against the OpenCV implementation of
the GrabCut interactive segmentation technique \cite{Rother:GrabCut04}, which
uses hand-tuned parameters instead of machine learning. On a standard dataset
\cite{Gulshan:CVPR10} our method learns higher order priors with hundreds of
parameter values, and produces significantly better segmentations. While our
focus is on binary labeling problems, we show that our techniques can be
naturally generalized to handle more than two labels
Half-integrality, LP-branching and FPT Algorithms
A recent trend in parameterized algorithms is the application of polytope
tools (specifically, LP-branching) to FPT algorithms (e.g., Cygan et al., 2011;
Narayanaswamy et al., 2012). However, although interesting results have been
achieved, the methods require the underlying polytope to have very restrictive
properties (half-integrality and persistence), which are known only for few
problems (essentially Vertex Cover (Nemhauser and Trotter, 1975) and Node
Multiway Cut (Garg et al., 1994)). Taking a slightly different approach, we
view half-integrality as a \emph{discrete} relaxation of a problem, e.g., a
relaxation of the search space from to such that
the new problem admits a polynomial-time exact solution. Using tools from CSP
(in particular Thapper and \v{Z}ivn\'y, 2012) to study the existence of such
relaxations, we provide a much broader class of half-integral polytopes with
the required properties, unifying and extending previously known cases.
In addition to the insight into problems with half-integral relaxations, our
results yield a range of new and improved FPT algorithms, including an
-time algorithm for node-deletion Unique Label Cover with
label set and an -time algorithm for Group Feedback Vertex
Set, including the setting where the group is only given by oracle access. All
these significantly improve on previous results. The latter result also implies
the first single-exponential time FPT algorithm for Subset Feedback Vertex Set,
answering an open question of Cygan et al. (2012).
Additionally, we propose a network flow-based approach to solve some cases of
the relaxation problem. This gives the first linear-time FPT algorithm to
edge-deletion Unique Label Cover.Comment: Added results on linear-time FPT algorithms (not present in SODA
paper
Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization
Creating impact in real-world settings requires artificial intelligence
techniques to span the full pipeline from data, to predictive models, to
decisions. These components are typically approached separately: a machine
learning model is first trained via a measure of predictive accuracy, and then
its predictions are used as input into an optimization algorithm which produces
a decision. However, the loss function used to train the model may easily be
misaligned with the end goal, which is to make the best decisions possible.
Hand-tuning the loss function to align with optimization is a difficult and
error-prone process (which is often skipped entirely).
We focus on combinatorial optimization problems and introduce a general
framework for decision-focused learning, where the machine learning model is
directly trained in conjunction with the optimization algorithm to produce
high-quality decisions. Technically, our contribution is a means of integrating
common classes of discrete optimization problems into deep learning or other
predictive models, which are typically trained via gradient descent. The main
idea is to use a continuous relaxation of the discrete problem to propagate
gradients through the optimization procedure. We instantiate this framework for
two broad classes of combinatorial problems: linear programs and submodular
maximization. Experimental results across a variety of domains show that
decision-focused learning often leads to improved optimization performance
compared to traditional methods. We find that standard measures of accuracy are
not a reliable proxy for a predictive model's utility in optimization, and our
method's ability to specify the true goal as the model's training objective
yields substantial dividends across a range of decision problems.Comment: Full version of paper accepted at AAAI 201
Structured learning of sum-of-submodular higher order energy functions
Submodular functions can be exactly minimized in polynomial time, and the special case that graph cuts solve with max flow [18] has had significant impact in computer vision [5, 20, 27]. In this paper we address the important class of sum-of-submodular (SoS) functions [2, 17], which can be efficiently minimized via a variant of max flow called submodular flow [6]. SoS functions can naturally express higher order priors involving, e.g., local image patches; however, it is difficult to fully exploit their expressive power because they have so many parameters. Rather than trying to formulate existing higher order priors as an SoS function, we take a discriminative learning approach, effectively searching the space of SoS functions for a higher order prior that performs well on our training set. We adopt a structural SVM approach [14, 33] and formulate the training problem in terms of quadratic programming; as a result we can efficiently search the space of SoS priors via an extended cutting-plane algorithm. We also show how the state-of-the-art max flow method for vision problems [10] can be modified to efficiently solve the submodular flow problem. Experimental comparisons are made against the OpenCV implementation of the GrabCut interactive segmentation technique [27], which uses hand-tuned parameters instead of machine learning. On a standard dataset [11] our method learns higher order priors with hundreds of parameter values, and produces significantly better segmentations. While our focus is on binary labeling problems, we show that our techniques can be naturally generalized to handle more than two labels. 1
The complexity of finite-valued CSPs
We study the computational complexity of exact minimisation of
rational-valued discrete functions. Let be a set of rational-valued
functions on a fixed finite domain; such a set is called a finite-valued
constraint language. The valued constraint satisfaction problem,
, is the problem of minimising a function given as
a sum of functions from . We establish a dichotomy theorem with respect
to exact solvability for all finite-valued constraint languages defined on
domains of arbitrary finite size.
We show that every constraint language either admits a binary
symmetric fractional polymorphism in which case the basic linear programming
relaxation solves any instance of exactly, or
satisfies a simple hardness condition that allows for a
polynomial-time reduction from Max-Cut to
Complexity of Discrete Energy Minimization Problems
Discrete energy minimization is widely-used in computer vision and machine
learning for problems such as MAP inference in graphical models. The problem,
in general, is notoriously intractable, and finding the global optimal solution
is known to be NP-hard. However, is it possible to approximate this problem
with a reasonable ratio bound on the solution quality in polynomial time? We
show in this paper that the answer is no. Specifically, we show that general
energy minimization, even in the 2-label pairwise case, and planar energy
minimization with three or more labels are exp-APX-complete. This finding rules
out the existence of any approximation algorithm with a sub-exponential
approximation ratio in the input size for these two problems, including
constant factor approximations. Moreover, we collect and review the
computational complexity of several subclass problems and arrange them on a
complexity scale consisting of three major complexity classes -- PO, APX, and
exp-APX, corresponding to problems that are solvable, approximable, and
inapproximable in polynomial time. Problems in the first two complexity classes
can serve as alternative tractable formulations to the inapproximable ones.
This paper can help vision researchers to select an appropriate model for an
application or guide them in designing new algorithms.Comment: ECCV'16 accepte
Valued Constraint Satisfaction Problems over Infinite Domains
The object of the thesis is the computational complexity of certain combinatorial optimisation problems called \emph{valued constraint satisfaction problems}, or \emph{VCSPs} for short. The requirements and optimisation criteria of these problems are expressed by sums of \emph{(valued) constraints} (also called \emph{cost functions}). More precisely, the input of a VCSP consists of a finite set of variables, a finite set of cost functions that depend on these variables, and a cost ; the task is to find values for the variables such that the sum of the cost functions is at most .
By restricting the set of possible cost functions in the input, a great variety of computational optimisation problems can be modelled as VCSPs. Recently, the computational complexity of all VCSPs for finite sets of cost functions over a finite domain has been classified. Many natural optimisation problems, however, cannot be formulated as VCSPs over a finite domain.
We initiate the systematic investigation of infinite-domain VCSPs by studying the complexity of VCSPs for piecewise linear (PL) and piecewise linear homogeneous (PLH) cost functions.
The VCSP for a finite set of PLH cost functions can be solved in polynomial time if the cost functions are improved by fully symmetric fractional operations of all arities. We
show this by (polynomial-time many-one) reducing the problem to a finite-domain VCSP which can be solved using a linear programming relaxation. We apply this result to show the polynomial-time tractability of VCSPs for {\it submodular} PLH cost functions, for {\it convex} PLH cost functions, and for {\it componentwise increasing} PLH cost functions; in fact, we show that submodular PLH functions and componentwise increasing PLH functions form maximally tractable classes of PLH cost functions.
We define the notion of {\it expressive power} for sets of cost functions over arbitrary domains, and discuss the relation between the expressive power and the set of fractional operations improving the same set of cost functions over an arbitrary countable domain.
Finally, we provide a polynomial-time algorithm solving the restriction of the VCSP for {\it all} PL cost functions to a fixed number of variables
The power of linear programming for general-valued CSPs
Let , called the domain, be a fixed finite set and let , called
the valued constraint language, be a fixed set of functions of the form
, where different functions might have
different arity . We study the valued constraint satisfaction problem
parametrised by , denoted by VCSP. These are minimisation
problems given by variables and the objective function given by a sum of
functions from , each depending on a subset of the variables.
Finite-valued constraint languages contain functions that take on only rational
values and not infinite values.
Our main result is a precise algebraic characterisation of valued constraint
languages whose instances can be solved exactly by the basic linear programming
relaxation (BLP). For a valued constraint language , BLP is a decision
procedure for if and only if admits a symmetric fractional
polymorphism of every arity. For a finite-valued constraint language ,
BLP is a decision procedure if and only if admits a symmetric
fractional polymorphism of some arity, or equivalently, if admits a
symmetric fractional polymorphism of arity 2.
Using these results, we obtain tractability of several novel classes of
problems, including problems over valued constraint languages that are: (1)
submodular on arbitrary lattices; (2) -submodular on arbitrary finite
domains; (3) weakly (and hence strongly) tree-submodular on arbitrary trees.Comment: A full version of a FOCS'12 paper by the last two authors
(arXiv:1204.1079) and an ICALP'13 paper by the first author (arXiv:1207.7213)
to appear in SIAM Journal on Computing (SICOMP
- …