1,263 research outputs found
Active-set Methods for Submodular Minimization Problems
International audienceWe consider the submodular function minimization (SFM) and the quadratic minimization problemsregularized by the Lov'asz extension of the submodular function. These optimization problemsare intimately related; for example,min-cut problems and total variation denoising problems, wherethe cut function is submodular and its Lov'asz extension is given by the associated total variation.When a quadratic loss is regularized by the total variation of a cut function, it thus becomes atotal variation denoising problem and we use the same terminology in this paper for “general” submodularfunctions. We propose a new active-set algorithm for total variation denoising with theassumption of an oracle that solves the corresponding SFM problem. This can be seen as localdescent algorithm over ordered partitions with explicit convergence guarantees. It is more flexiblethan the existing algorithms with the ability for warm-restarts using the solution of a closely relatedproblem. Further, we also consider the case when a submodular function can be decomposed intothe sum of two submodular functions F1 and F2 and assume SFM oracles for these two functions.We propose a new active-set algorithm for total variation denoising (and hence SFM by thresholdingthe solution at zero). This algorithm also performs local descent over ordered partitions and itsability to warm start considerably improves the performance of the algorithm. In the experiments,we compare the performance of the proposed algorithms with state-of-the-art algorithms, showingthat it reduces the calls to SFM oracles
Structured Sparsity: Discrete and Convex approaches
Compressive sensing (CS) exploits sparsity to recover sparse or compressible
signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity
is also used to enhance interpretability in machine learning and statistics
applications: While the ambient dimension is vast in modern data analysis
problems, the relevant information therein typically resides in a much lower
dimensional space. However, many solutions proposed nowadays do not leverage
the true underlying structure. Recent results in CS extend the simple sparsity
idea to more sophisticated {\em structured} sparsity models, which describe the
interdependency between the nonzero components of a signal, allowing to
increase the interpretability of the results and lead to better recovery
performance. In order to better understand the impact of structured sparsity,
in this chapter we analyze the connections between the discrete models and
their convex relaxations, highlighting their relative advantages. We start with
the general group sparse model and then elaborate on two important special
cases: the dispersive and the hierarchical models. For each, we present the
models in their discrete nature, discuss how to solve the ensuing discrete
problems and then describe convex relaxations. We also consider more general
structures as defined by set functions and present their convex proxies.
Further, we discuss efficient optimization solutions for structured sparsity
problems and illustrate structured sparsity in action via three applications.Comment: 30 pages, 18 figure
A Combinatorial, Strongly Polynomial-Time Algorithm for Minimizing Submodular Functions
This paper presents the first combinatorial polynomial-time algorithm for
minimizing submodular set functions, answering an open question posed in 1981
by Grotschel, Lovasz, and Schrijver. The algorithm employs a scaling scheme
that uses a flow in the complete directed graph on the underlying set with each
arc capacity equal to the scaled parameter. The resulting algorithm runs in
time bounded by a polynomial in the size of the underlying set and the largest
length of the function value. The paper also presents a strongly
polynomial-time version that runs in time bounded by a polynomial in the size
of the underlying set independent of the function value.Comment: 17 page
On the Convergence Rate of Decomposable Submodular Function Minimization
Submodular functions describe a variety of discrete problems in machine
learning, signal processing, and computer vision. However, minimizing
submodular functions poses a number of algorithmic challenges. Recent work
introduced an easy-to-use, parallelizable algorithm for minimizing submodular
functions that decompose as the sum of "simple" submodular functions.
Empirically, this algorithm performs extremely well, but no theoretical
analysis was given. In this paper, we show that the algorithm converges
linearly, and we provide upper and lower bounds on the rate of convergence. Our
proof relies on the geometry of submodular polyhedra and draws on results from
spectral graph theory.Comment: 17 pages, 3 figure
- …