42 research outputs found
Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions
We investigate three related and important problems connected to machine
learning: approximating a submodular function everywhere, learning a submodular
function (in a PAC-like setting [53]), and constrained minimization of
submodular functions. We show that the complexity of all three problems depends
on the 'curvature' of the submodular function, and provide lower and upper
bounds that refine and improve previous results [3, 16, 18, 52]. Our proof
techniques are fairly generic. We either use a black-box transformation of the
function (for approximation and learning), or a transformation of algorithms to
use an appropriate surrogate function (for minimization). Curiously, curvature
has been known to influence approximations for submodular maximization [7, 55],
but its effect on minimization, approximation and learning has hitherto been
open. We complete this picture, and also support our theoretical claims by
empirical results.Comment: 21 pages. A shorter version appeared in Advances of NIPS-201
Density Functions subject to a Co-Matroid Constraint
In this paper we consider the problem of finding the {\em densest} subset
subject to {\em co-matroid constraints}. We are given a {\em monotone
supermodular} set function defined over a universe , and the density of
a subset is defined to be f(S)/\crd{S}. This generalizes the concept of
graph density. Co-matroid constraints are the following: given matroid \calM
a set is feasible, iff the complement of is {\em independent} in the
matroid. Under such constraints, the problem becomes \np-hard. The specific
case of graph density has been considered in literature under specific
co-matroid constraints, for example, the cardinality matroid and the partition
matroid. We show a 2-approximation for finding the densest subset subject to
co-matroid constraints. Thus, for instance, we improve the approximation
guarantees for the result for partition matroids in the literature
Maximizing Symmetric Submodular Functions
Symmetric submodular functions are an important family of submodular
functions capturing many interesting cases including cut functions of graphs
and hypergraphs. Maximization of such functions subject to various constraints
receives little attention by current research, unlike similar minimization
problems which have been widely studied. In this work, we identify a few
submodular maximization problems for which one can get a better approximation
for symmetric objectives than the state of the art approximation for general
submodular functions.
We first consider the problem of maximizing a non-negative symmetric
submodular function subject to a
down-monotone solvable polytope . For
this problem we describe an algorithm producing a fractional solution of value
at least , where is the optimal integral solution.
Our second result considers the problem for a
non-negative symmetric submodular function . For this problem, we give an approximation ratio that depends on
the value and is always at least . Our method can
also be applied to non-negative non-symmetric submodular functions, in which
case it produces approximation, improving over the best known
result for this problem. For unconstrained maximization of a non-negative
symmetric submodular function we describe a deterministic linear-time
-approximation algorithm. Finally, we give a -approximation algorithm for Submodular Welfare with players having
identical non-negative submodular utility functions, and show that this is the
best possible approximation ratio for the problem.Comment: 31 pages, an extended abstract appeared in ESA 201
Knapsack Cover Subject to a Matroid Constraint
We consider the Knapsack Covering problem subject to a matroid constraint. In this problem, we are given an universe U of n items where item i has attributes: a cost c(i) and a size s(i). We also have a demand D. We are also given a matroid M = (U, I) on the set U. A feasible solution S to the problem is one such that (i) the cumulative size of the items chosen is at least D, and (ii) the set S is independent in the matroid M (i.e. S is in I). The objective is to minimize the total cost of the items selected, sum_{i in S}c(i).
Our main result proves a 2-factor approximation for this problem.
The problem described above falls in the realm of mixed packing covering problems. We also consider packing extensions of certain other covering problems and prove that in such cases it is not possible to derive any constant factor pproximations
Contributions on secretary problems, independent sets of rectangles and related problems
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mathematics, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 187-198).We study three problems arising from different areas of combinatorial optimization. We first study the matroid secretary problem, which is a generalization proposed by Babaioff, Immorlica and Kleinberg of the classical secretary problem. In this problem, the elements of a given matroid are revealed one by one. When an element is revealed, we learn information about its weight and decide to accept it or not, while keeping the accepted set independent in the matroid. The goal is to maximize the expected weight of our solution. We study different variants for this problem depending on how the elements are presented and on how the weights are assigned to the elements. Our main result is the first constant competitive algorithm for the random-assignment random-order model. In this model, a list of hidden nonnegative weights is randomly assigned to the elements of the matroid, which are later presented to us in uniform random order, independent of the assignment. The second problem studied is the jump number problem. Consider a linear extension L of a poset P. A jump is a pair of consecutive elements in L that are not comparable in P. Finding a linear extension minimizing the number of jumps is NP-hard even for chordal bipartite posets. For the class of posets having two directional orthogonal ray comparability graphs, we show that this problem is equivalent to finding a maximum independent set of a well-behaved family of rectangles. Using this, we devise combinatorial and LP-based algorithms for the jump number problem, extending the class of bipartite posets for which this problem is polynomially solvable and improving on the running time of existing algorithms for certain subclasses. The last problem studied is the one of finding nonempty minimizers of a symmetric submodular function over any family of sets closed under inclusion. We give an efficient O(ns)-time algorithm for this task, based on Queyranne's pendant pair technique for minimizing unconstrained symmetric submodular functions. We extend this algorithm to report all inclusion-wise nonempty minimal minimizers under hereditary constraints of slightly more general functions.by José Antonio Soto.Ph.D
Randomized Contractions for Multiobjective Minimum Cuts
We show that Karger\u27s randomized contraction method (SODA 93) can be adapted to multiobjective global minimum cut problems with a constant number of edge or node budget constraints to give efficient algorithms.
For global minimum cuts with a single edge-budget constraint, our extension of the randomized contraction method has running time tilde{O}(n^3) in an n-node graph improving upon the best-known randomized algorithm with running time tilde{O}(n^4) due to Armon and Zwick (Algorithmica 2006). Our analysis also gives a new upper bound of O(n^3) for the number of optimal solutions for a single edge-budget min cut problem. For the case of (k-1) edge-budget constraints, the extension of our algorithm saves a logarithmic factor from the best-known randomized running time of O(n^{2k} log^3 n). A main feature of our algorithms is to adaptively choose, at each step, the appropriate cost function used in the random selection of edges to be contracted.
For the global min cut problem with a constant number of node budgets, we give a randomized algorithm with running time tilde{O}(n^2), improving the current best determinisitic running time of O(n^3) due to Goemans and Soto (SIAM Journal on Discrete Mathematics 2013). Our method also shows that the total number of distinct optimal solutions is bounded by O(n^2) as in the case of global min-cuts. Our algorithm extends to the node-budget constrained global min cut problem excluding a given sink with the same running time and bound on number of optimal solutions, again improving upon the best-known running time by a factor of O(n). For node-budget constrained problems, our improvements arise from incorporating the idea of merging any infeasible super-nodes that arise during the random contraction process.
In contrast to cuts excluding a sink, we note that the node-cardinality constrained min-cut problem containing a given source is strongly NP-hard using a reduction from graph bisection
Scheduling to minimize power consumption using submodular functions
Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 59-64).We develop logarithmic approximation algorithms for extremely general formulations of multiprocessor multi-interval offline task scheduling to minimize power usage. Here each processor has an arbitrary specified power consumption to be turned on for each possible time interval, and each job has a specified list of time interval/processor pairs during which it could be scheduled. (A processor need not be in use for an entire interval it is turned on.) If there is a feasible schedule, our algorithm finds a feasible schedule with total power usage within an O(log n) factor of optimal, where n is the number of jobs. (Even in a simple setting with one processor, the problem is Set-Cover hard.) If not all jobs can be scheduled and each job has a specified value, then our algorithm finds a schedule of value at least (1 - c)Z and power usage within an O(log(1/E)) factor of the optimal schedule of value at least Z, for any specified Z and c > 0. At the foundation of our work is a general framework for logarithmic approximation to maximizing any submodular function subject to budget constraints. We also introduce the online version of this scheduling problem, and show its relation to the classical secretary problem. In order to obtain constant competitive algorithms for this online version, we study the secretary problem with submodular utility function. We present several constant competitive algorithms for the secretary problem with different kinds of utility functions.by Morteza Zadimoghaddam.S.M
Learning with Submodular Functions: A Convex Optimization Perspective
International audienceSubmodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular functions and (2) the lovasz extension of submodular functions provides a useful set of regularization functions for supervised and unsupervised learning. In this monograph, we present the theory of submodular functions from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems. In particular, we show how submodular function minimization is equivalent to solving a wide variety of convex optimization problems. This allows the derivation of new efficient algorithms for approximate and exact submodular function minimization with theoretical guarantees and good practical performance. By listing many examples of submodular functions, we review various applications to machine learning, such as clustering, experimental design, sensor placement, graphical model structure learning or subset selection, as well as a family of structured sparsity-inducing norms that can be derived and used from submodular functions