Greed is Still Good: Maximizing Monotone Submodular+Supermodular Functions

We analyze the performance of the greedy algorithm, and also a discrete semi-gradient based algorithm, for maximizing the sum of a suBmodular and suPermodular (BP) function (both of which are non-negative monotone non-decreasing) under two types of constraints, either a cardinality constraint or $p\geq 1$ matroid independence constraints. These problems occur naturally in several real-world applications in data science, machine learning, and artificial intelligence. The problems are ordinarily inapproximable to any factor (as we show). Using the curvature $\kappa_f$ of the submodular term, and introducing $\kappa^g$ for the supermodular term (a natural dual curvature for supermodular functions), however, both of which are computable in linear time, we show that BP maximization can be efficiently approximated by both the greedy and the semi-gradient based algorithm. The algorithms yield multiplicative guarantees of $\frac{1}{\kappa_f}\left[1-e^{-(1-\kappa^g)\kappa_f}\right]$ and $\frac{1-\kappa^g}{(1-\kappa^g)\kappa_f + p}$ for the two types of constraints respectively. For pure monotone supermodular constrained maximization, these yield $1-\kappa^g$ and $(1-\kappa^g)/p$ for the two types of constraints respectively. We also analyze the hardness of BP maximization and show that our guarantees match hardness by a constant factor and by $O(\ln(p))$ respectively. Computational experiments are also provided supporting our analysis

Non-submodular Function Maximization subject to a Matroid Constraint, with Applications

The standard greedy algorithm has been recently shown to enjoy approximation guarantees for constrained non-submodular nondecreasing set function maximization. While these recent results allow to better characterize the empirical success of the greedy algorithm, they are only applicable to simple cardinality constraints. In this paper, we study the problem of maximizing a non-submodular nondecreasing set function subject to a general matroid constraint. We first show that the standard greedy algorithm offers an approximation factor of $\frac{0.4 {\gamma}^{2}}{\sqrt{\gamma r} + 1}$, where $\gamma$ is the submodularity ratio of the function and $r$ is the rank of the matroid. Then, we show that the same greedy algorithm offers a constant approximation factor of $(1 + 1/(1-\alpha))^{-1}$, where $\alpha$ is the generalized curvature of the function. In addition, we demonstrate that these approximation guarantees are applicable to several real-world applications in which the submodularity ratio and the generalized curvature can be bounded. Finally, we show that our greedy algorithm does achieve a competitive performance in practice using a variety of experiments on synthetic and real-world data.Comment: Added missing citations and changed strong submodularity ratio to generalized curvatur

Submodular Optimization Problems and Greedy Strategies: A Survey

The greedy strategy is an approximation algorithm to solve optimization problems arising in decision making with multiple actions. How good is the greedy strategy compared to the optimal solution? In this survey, we mainly consider two classes of optimization problems where the objective function is submodular. The first is set submodular optimization, which is to choose a set of actions to optimize a set submodular objective function, and the second is string submodular optimization, which is to choose an ordered set of actions to optimize a string submodular function. Our emphasis here is on performance bounds for the greedy strategy in submodular optimization problems. Specifically, we review performance bounds for the greedy strategy, more general and improved bounds in terms of curvature, performance bounds for the batched greedy strategy, and performance bounds for Nash equilibria

Polyhedral aspects of Submodularity, Convexity and Concavity

Seminal work by Edmonds and Lovasz shows the strong connection between submodularity and convexity. Submodular functions have tight modular lower bounds, and subdifferentials in a manner akin to convex functions. They also admit poly-time algorithms for minimization and satisfy the Fenchel duality theorem and the Discrete Seperation Theorem, both of which are fundamental characteristics of convex functions. Submodular functions also show signs similar to concavity. Submodular maximization, though NP hard, admits constant factor approximation guarantees. Concave functions composed with modular functions are submodular, and they also satisfy diminishing returns property. This manuscript provides a more complete picture on the relationship between submodularity with convexity and concavity, by extending many of the results connecting submodularity with convexity to the concave aspects of submodularity. We first show the existence of superdifferentials, and efficiently computable tight modular upper bounds of a submodular function. While we show that it is hard to characterize this polyhedron, we obtain inner and outer bounds on the superdifferential along with certain specific and useful supergradients. We then investigate forms of concave extensions of submodular functions and show interesting relationships to submodular maximization. We next show connections between optimality conditions over the superdifferentials and submodular maximization, and show how forms of approximate optimality conditions translate into approximation factors for maximization. We end this paper by studying versions of the discrete seperation theorem and the Fenchel duality theorem when seen from the concave point of view. In every case, we relate our results to the existing results from the convex point of view, thereby improving the analysis of the relationship between submodularity, convexity, and concavity.Comment: 38 pages, 10 figure

Dependent Randomized Rounding for Matroid Polytopes and Applications

Motivated by several applications, we consider the problem of randomly rounding a fractional solution in a matroid (base) polytope to an integral one. We consider the pipage rounding technique and also present a new technique, randomized swap rounding. Our main technical results are concentration bounds for functions of random variables arising from these rounding techniques. We prove Chernoff-type concentration bounds for linear functions of random variables arising from both techniques, and also a lower-tail exponential bound for monotone submodular functions of variables arising from randomized swap rounding. The following are examples of our applications: (1) We give a (1-1/e-epsilon)-approximation algorithm for the problem of maximizing a monotone submodular function subject to 1 matroid and k linear constraints, for any constant k and epsilon>0. (2) We present a result on minimax packing problems that involve a matroid base constraint. We give an O(log m / log log m)-approximation for the general problem Min {lambda: x \in {0,1}^N, x \in B(M), Ax <= lambda b}, where m is the number of packing constraints. (3) We generalize the continuous greedy algorithm to problems involving multiple submodular functions, and use it to find a (1-1/e-epsilon)-approximate pareto set for the problem of maximizing a constant number of monotone submodular functions subject to a matroid constraint. An example is the Submodular Welfare Problem where we are looking for an approximate pareto set with respect to individual players' utilities.Comment: Rico Zenklusen joined as an author; paper substantially expanded compared to previous version; note a slight change in the titl

Performance Bounds for the kk-Batch Greedy Strategy in Optimization Problems with Curvature

The $k$-batch greedy strategy is an approximate algorithm to solve optimization problems where the optimal solution is hard to obtain. Starting with the empty set, the $k$-batch greedy strategy adds a batch of $k$ elements to the current solution set with the largest gain in the objective function while satisfying the constraints. In this paper, we bound the performance of the $k$-batch greedy strategy with respect to the optimal strategy by defining the total curvature $\alpha_k$. We show that when the objective function is nondecreasing and submodular, the $k$-batch greedy strategy satisfies a harmonic bound $1/(1+\alpha_k)$ for a general matroid constraint and an exponential bound $\left(1-(1-{\alpha}_k/{t})^t\right)/{\alpha}_k$ for a uniform matroid constraint, where $k$ divides the cardinality of the maximal set in the general matroid, $t=K/k$ is an integer, and $K$ is the rank of the uniform matroid. We also compare the performance of the $k$-batch greedy strategy with that of the $k_1$-batch greedy strategy when $k_1$ divides $k$. Specifically, we prove that when the objective function is nondecreasing and submodular, the $k$-batch greedy strategy has better harmonic and exponential bounds in terms of the total curvature. Finally, we illustrate our results by considering a task-assignment problem.Comment: This paper has been accepted by 2016 AC

Complete enumeration of small realizable oriented matroids

Enumeration of all combinatorial types of point configurations and polytopes is a fundamental problem in combinatorial geometry. Although many studies have been done, most of them are for 2-dimensional and non-degenerate cases. Finschi and Fukuda (2001) published the first database of oriented matroids including degenerate (i.e. non-uniform) ones and of higher ranks. In this paper, we investigate algorithmic ways to classify them in terms of realizability, although the underlying decision problem of realizability checking is NP-hard. As an application, we determine all possible combinatorial types (including degenerate ones) of 3-dimensional configurations of 8 points, 2-dimensional configurations of 9 points and 5-dimensional configurations of 9 points. We could also determine all possible combinatorial types of 5-polytopes with 9 vertices.Comment: 19 pages, 2 figure

Optimization with More than One Budget

A natural way to deal with multiple, partially conflicting objectives is turning all the objectives but one into budget constraints. Some classical polynomial-time optimization problems, such as spanning tree and forest, shortest path, (perfect) matching, independent set (basis) in a matroid or in the intersection of two matroids, become NP-hard even with one budget constraint. Still, for most of these problems deterministic and randomized polynomial-time approximation schemes are known. In the case of two or more budgets, typically only multi-criteria approximation schemes are available, which return slightly infeasible solutions. Not much is known however for the case of strict budget constraints: filling this gap is the main goal of this paper. We show that shortest path, perfect matching, and spanning tree (and hence matroid basis and matroid intersection basis) are inapproximable already with two budget constraints. For the remaining problems, whose set of solutions forms an independence system, we present deterministic and randomized polynomial-time approximation schemes for a constant number k of budget constraints. Our results are based on a variety of techniques: 1. We present a simple and powerful mechanism to transform multi-criteria approximation schemes into pure approximation schemes. 2. We show that points in low dimensional faces of any matroid polytope are almost integral, an interesting result on its own. This gives a deterministic approximation scheme for k-budgeted matroid independent set. 3. We present a deterministic approximation scheme for 2-budgeted matching. The backbone of this result is a purely topological property of curves in R^2

Submodular Function Maximization over Distributive and Integer Lattices

The problem of maximizing non-negative submodular functions has been studied extensively in the last few years. However, most papers consider submodular set functions. Recently, several advances have been made for the more general case of submodular functions on the integer lattice. In this paper, we present a deterministic $\frac{1}{3}$-approximation for maximizing a submodular function on a bounded integer lattice $\{0, \ldots, C\}^n$ using a Double Greedy framework. Moreover, we show that the analysis is tight and that other ideas used for approximating set functions cannot easily be extended. In contrast to set functions, submodularity on the integer lattice does not imply the so-called diminishing returns property. Assuming this property, it was shown that many results for set functions can also be obtained for the integer lattice. In this paper, we consider a further generalization. Instead of the integer lattice, we consider a distributive lattice as the function domain and assume the diminishing returns (DR) property. On the one hand, we show that some approximation algorithms match the set functions setting. In particular, we can obtain a $\frac{1}{2}$-approximation for unconstrained maximization, a $(1-\frac{1}{e})$-approximation for monotone functions under a cardinality constraint and a $\frac{1}{2}$-approximation for a poset matroid constraint. On the other hand, for a knapsack constraint, the problem becomes significantly harder: even for monotone DR-submodular functions, we show that there is no $2^{(\log (n^{1/2} - 1))^\delta - 1}$-approximation for every $\delta > 0$ under the assumption that $3-SAT$ cannot be solved in time $2^{n^{3/4 + \epsilon}}$

Combinatorial Optimization Problems with Interaction Costs: Complexity and Solvable Cases

We introduce and study the combinatorial optimization problem with interaction costs (COPIC). COPIC is the problem of finding two combinatorial structures, one from each of two given families, such that the sum of their independent linear costs and the interaction costs between elements of the two selected structures is minimized. COPIC generalizes the quadratic assignment problem and many other well studied combinatorial optimization problems, and hence covers many real world applications. We show how various topics from different areas in the literature can be formulated as special cases of COPIC. The main contributions of this paper are results on the computational complexity and approximability of COPIC for different families of combinatorial structures (e.g. spanning trees, paths, matroids), and special structures of the interaction costs. More specifically, we analyze the complexity if the interaction cost matrix is parameterized by its rank and if it is a diagonal matrix. Also, we determine the structure of the intersection cost matrix, such that COPIC is equivalent to independently solving linear optimization problems for the two given families of combinatorial structures