1,004 research outputs found
Reflection methods for user-friendly submodular optimization
Recently, it has become evident that submodularity naturally captures widely
occurring concepts in machine learning, signal processing and computer vision.
Consequently, there is need for efficient optimization procedures for
submodular functions, especially for minimization problems. While general
submodular minimization is challenging, we propose a new method that exploits
existing decomposability of submodular functions. In contrast to previous
approaches, our method is neither approximate, nor impractical, nor does it
need any cumbersome parameter tuning. Moreover, it is easy to implement and
parallelize. A key component of our method is a formulation of the discrete
submodular minimization problem as a continuous best approximation problem that
is solved through a sequence of reflections, and its solution can be easily
thresholded to obtain an optimal discrete solution. This method solves both the
continuous and discrete formulations of the problem, and therefore has
applications in learning, inference, and reconstruction. In our experiments, we
illustrate the benefits of our method on two image segmentation tasks.Comment: Neural Information Processing Systems (NIPS), \'Etats-Unis (2013
A Combinatorial, Strongly Polynomial-Time Algorithm for Minimizing Submodular Functions
This paper presents the first combinatorial polynomial-time algorithm for
minimizing submodular set functions, answering an open question posed in 1981
by Grotschel, Lovasz, and Schrijver. The algorithm employs a scaling scheme
that uses a flow in the complete directed graph on the underlying set with each
arc capacity equal to the scaled parameter. The resulting algorithm runs in
time bounded by a polynomial in the size of the underlying set and the largest
length of the function value. The paper also presents a strongly
polynomial-time version that runs in time bounded by a polynomial in the size
of the underlying set independent of the function value.Comment: 17 page
Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions
We investigate three related and important problems connected to machine
learning: approximating a submodular function everywhere, learning a submodular
function (in a PAC-like setting [53]), and constrained minimization of
submodular functions. We show that the complexity of all three problems depends
on the 'curvature' of the submodular function, and provide lower and upper
bounds that refine and improve previous results [3, 16, 18, 52]. Our proof
techniques are fairly generic. We either use a black-box transformation of the
function (for approximation and learning), or a transformation of algorithms to
use an appropriate surrogate function (for minimization). Curiously, curvature
has been known to influence approximations for submodular maximization [7, 55],
but its effect on minimization, approximation and learning has hitherto been
open. We complete this picture, and also support our theoretical claims by
empirical results.Comment: 21 pages. A shorter version appeared in Advances of NIPS-201
Minimizing a sum of submodular functions
We consider the problem of minimizing a function represented as a sum of
submodular terms. We assume each term allows an efficient computation of {\em
exchange capacities}. This holds, for example, for terms depending on a small
number of variables, or for certain cardinality-dependent terms.
A naive application of submodular minimization algorithms would not exploit
the existence of specialized exchange capacity subroutines for individual
terms. To overcome this, we cast the problem as a {\em submodular flow} (SF)
problem in an auxiliary graph, and show that applying most existing SF
algorithms would rely only on these subroutines.
We then explore in more detail Iwata's capacity scaling approach for
submodular flows (Math. Programming, 76(2):299--308, 1997). In particular, we
show how to improve its complexity in the case when the function contains
cardinality-dependent terms.Comment: accepted to "Discrete Applied Mathematics
Structured learning of sum-of-submodular higher order energy functions
Submodular functions can be exactly minimized in polynomial time, and the
special case that graph cuts solve with max flow \cite{KZ:PAMI04} has had
significant impact in computer vision
\cite{BVZ:PAMI01,Kwatra:SIGGRAPH03,Rother:GrabCut04}. In this paper we address
the important class of sum-of-submodular (SoS) functions
\cite{Arora:ECCV12,Kolmogorov:DAM12}, which can be efficiently minimized via a
variant of max flow called submodular flow \cite{Edmonds:ADM77}. SoS functions
can naturally express higher order priors involving, e.g., local image patches;
however, it is difficult to fully exploit their expressive power because they
have so many parameters. Rather than trying to formulate existing higher order
priors as an SoS function, we take a discriminative learning approach,
effectively searching the space of SoS functions for a higher order prior that
performs well on our training set. We adopt a structural SVM approach
\cite{Joachims/etal/09a,Tsochantaridis/etal/04} and formulate the training
problem in terms of quadratic programming; as a result we can efficiently
search the space of SoS priors via an extended cutting-plane algorithm. We also
show how the state-of-the-art max flow method for vision problems
\cite{Goldberg:ESA11} can be modified to efficiently solve the submodular flow
problem. Experimental comparisons are made against the OpenCV implementation of
the GrabCut interactive segmentation technique \cite{Rother:GrabCut04}, which
uses hand-tuned parameters instead of machine learning. On a standard dataset
\cite{Gulshan:CVPR10} our method learns higher order priors with hundreds of
parameter values, and produces significantly better segmentations. While our
focus is on binary labeling problems, we show that our techniques can be
naturally generalized to handle more than two labels
Discrete Convex Functions on Graphs and Their Algorithmic Applications
The present article is an exposition of a theory of discrete convex functions
on certain graph structures, developed by the author in recent years. This
theory is a spin-off of discrete convex analysis by Murota, and is motivated by
combinatorial dualities in multiflow problems and the complexity classification
of facility location problems on graphs. We outline the theory and algorithmic
applications in combinatorial optimization problems
- …