Reflection methods for user-friendly submodular optimization

Recently, it has become evident that submodularity naturally captures widely occurring concepts in machine learning, signal processing and computer vision. Consequently, there is need for efficient optimization procedures for submodular functions, especially for minimization problems. While general submodular minimization is challenging, we propose a new method that exploits existing decomposability of submodular functions. In contrast to previous approaches, our method is neither approximate, nor impractical, nor does it need any cumbersome parameter tuning. Moreover, it is easy to implement and parallelize. A key component of our method is a formulation of the discrete submodular minimization problem as a continuous best approximation problem that is solved through a sequence of reflections, and its solution can be easily thresholded to obtain an optimal discrete solution. This method solves both the continuous and discrete formulations of the problem, and therefore has applications in learning, inference, and reconstruction. In our experiments, we illustrate the benefits of our method on two image segmentation tasks.Comment: Neural Information Processing Systems (NIPS), \'Etats-Unis (2013

Combinatorial Continuous Maximal Flows

Maximum flow (and minimum cut) algorithms have had a strong impact on computer vision. In particular, graph cuts algorithms provide a mechanism for the discrete optimization of an energy functional which has been used in a variety of applications such as image segmentation, stereo, image stitching and texture synthesis. Algorithms based on the classical formulation of max-flow defined on a graph are known to exhibit metrication artefacts in the solution. Therefore, a recent trend has been to instead employ a spatially continuous maximum flow (or the dual min-cut problem) in these same applications to produce solutions with no metrication errors. However, known fast continuous max-flow algorithms have no stopping criteria or have not been proved to converge. In this work, we revisit the continuous max-flow problem and show that the analogous discrete formulation is different from the classical max-flow problem. We then apply an appropriate combinatorial optimization technique to this combinatorial continuous max-flow CCMF problem to find a null-divergence solution that exhibits no metrication artefacts and may be solved exactly by a fast, efficient algorithm with provable convergence. Finally, by exhibiting the dual problem of our CCMF formulation, we clarify the fact, already proved by Nozawa in the continuous setting, that the max-flow and the total variation problems are not always equivalent.Comment: 26 page

Maximum flows and minimum cuts in the plane

A continuous maximum flow problem finds the largest t such that div v = t F(x, y) is possible with a capacity constraint ||(v[subscript 1], v[subscript 2])|| ≤ c(x, y). The dual problem finds a minimum cut ∂ S which is filled to capacity by the flow through it. This model problem has found increasing application in medical imaging, and the theory continues to develop (along with new algorithms). Remaining difficulties include explicit streamlines for the maximum flow, and constraints that are analogous to a directed graph. Keywords: Maximum flow; Minimum cut; Capacity constraint; Cheege

A node-capacitated Okamura-Seymour theorem

The classical Okamura-Seymour theorem states that for an edge-capacitated, multi-commodity flow instance in which all terminals lie on a single face of a planar graph, there exists a feasible concurrent flow if and only if the cut conditions are satisfied. Simple examples show that a similar theorem is impossible in the node-capacitated setting. Nevertheless, we prove that an approximate flow/cut theorem does hold: For some universal c > 0, if the node cut conditions are satisfied, then one can simultaneously route a c-fraction of all the demands. This answers an open question of Chekuri and Kawarabayashi. More generally, we show that this holds in the setting of multi-commodity polymatroid networks introduced by Chekuri, et. al. Our approach employs a new type of random metric embedding in order to round the convex programs corresponding to these more general flow problems.Comment: 30 pages, 5 figure

Robust Flows over Time: Models and Complexity Results

We study dynamic network flows with uncertain input data under a robust optimization perspective. In the dynamic maximum flow problem, the goal is to maximize the flow reaching the sink within a given time horizon $T$, while flow requires a certain travel time to traverse an edge. In our setting, we account for uncertain travel times of flow. We investigate maximum flows over time under the assumption that at most $\Gamma$ travel times may be prolonged simultaneously due to delay. We develop and study a mathematical model for this problem. As the dynamic robust flow problem generalizes the static version, it is NP-hard to compute an optimal flow. However, our dynamic version is considerably more complex than the static version. We show that it is NP-hard to verify feasibility of a given candidate solution. Furthermore, we investigate temporally repeated flows and show that in contrast to the non-robust case (that is, without uncertainties) they no longer provide optimal solutions for the robust problem, but rather yield a worst case optimality gap of at least $T$. We finally show that the optimality gap is at most $O(\eta k \log T)$, where $\eta$ and $k$ are newly introduced instance characteristics and provide a matching lower bound instance with optimality gap $\Omega(\log T)$ and $\eta = k = 1$. The results obtained in this paper yield a first step towards understanding robust dynamic flow problems with uncertain travel times

Submodular relaxation for inference in Markov random fields

In this paper we address the problem of finding the most probable state of a discrete Markov random field (MRF), also known as the MRF energy minimization problem. The task is known to be NP-hard in general and its practical importance motivates numerous approximate algorithms. We propose a submodular relaxation approach (SMR) based on a Lagrangian relaxation of the initial problem. Unlike the dual decomposition approach of Komodakis et al., 2011 SMR does not decompose the graph structure of the initial problem but constructs a submodular energy that is minimized within the Lagrangian relaxation. Our approach is applicable to both pairwise and high-order MRFs and allows to take into account global potentials of certain types. We study theoretical properties of the proposed approach and evaluate it experimentally.Comment: This paper is accepted for publication in IEEE Transactions on Pattern Analysis and Machine Intelligenc

Barrier Frank-Wolfe for Marginal Inference

We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This modular structure enables us to leverage black-box MAP solvers (both exact and approximate) for variational inference, and obtains more accurate results than tree-reweighted algorithms that optimize over the local consistency relaxation. Theoretically, we bound the sub-optimality for the proposed algorithm despite the TRW objective having unbounded gradients at the boundary of the marginal polytope. Empirically, we demonstrate the increased quality of results found by tightening the relaxation over the marginal polytope as well as the spanning tree polytope on synthetic and real-world instances.Comment: 25 pages, 12 figures, To appear in Neural Information Processing Systems (NIPS) 2015, Corrected reference and cleaned up bibliograph

Inexactness of SDP Relaxation and Valid Inequalities for Optimal Power Flow

It has been recently proven that the semidefinite programming (SDP) relaxation of the optimal power flow problem over radial networks is exact under technical conditions such as not including generation lower bounds or allowing load over-satisfaction. In this paper, we investigate the situation where generation lower bounds are present. We show that even for a two-bus one-generator system, the SDP relaxation can have all possible approximation outcomes, that is (1) SDP relaxation may be exact or (2) SDP relaxation may be inexact or (3) SDP relaxation may be feasible while the OPF instance may be infeasible. We provide a complete characterization of when these three approximation outcomes occur and an analytical expression of the resulting optimality gap for this two-bus system. In order to facilitate further research, we design a library of instances over radial networks in which the SDP relaxation has positive optimality gap. Finally, we propose valid inequalities and variable bound tightening techniques that significantly improve the computational performance of a global optimization solver. Our work demonstrates the need of developing efficient global optimization methods for the solution of OPF even in the simple but fundamental case of radial networks

Robust network design under polyhedral traffic uncertainty

Ankara : The Department of Industrial Engineering and The Institute of Engineering and Science of Bilkent Univ., 2007.Thesis (Ph.D.) -- Bilkent University, 2007.Includes bibliographical references leaves 160-166.In this thesis, we study the design of networks robust to changes in demand estimates. We consider the case where the set of feasible demands is defined by an arbitrary polyhedron. Our motivation is to determine link capacity or routing configurations, which remain feasible for any realization in the corresponding demand polyhedron. We consider three well-known problems under polyhedral demand uncertainty all of which are posed as semi-infinite mixed integer programming problems. We develop explicit, compact formulations for all three problems as well as alternative formulations and exact solution methods. The first problem arises in the Virtual Private Network (VPN) design field. We present compact linear mixed-integer programming formulations for the problem with the classical hose traffic model and for a new, less conservative, robust variant relying on accessible traffic statistics. Although we can solve these formulations for medium-to-large instances in reasonable times using off-the-shelf MIP solvers, we develop a combined branch-and-price and cutting plane algorithm to handle larger instances. We also provide an extensive discussion of our numerical results. Next, we study the Open Shortest Path First (OSPF) routing enhanced with traffic engineering tools under general demand uncertainty with the motivation to discuss if OSPF could be made comparable to the general unconstrained routing (MPLS) when it is provided with a less restrictive operating environment. To the best of our knowledge, these two routing mechanisms are compared for the first time under such a general setting. We provide compact formulations for both routing types and show that MPLS routing for polyhedral demands can be computed in polynomial time. Moreover, we present a specialized branchand-price algorithm strengthened with the inclusion of cuts as an exact solution tool. Subsequently, we compare the new and more flexible OSPF routing with MPLS as well as the traditional OSPF on several network instances. We observe that the management tools we use in OSPF make it significantly better than the generic OSPF. Moreover, we show that OSPF performance can get closer to that of MPLS in some cases. Finally, we consider the Network Loading Problem (NLP) under a polyhedral uncertainty description of traffic demands. After giving a compact multicommodity formulation of the problem, we prove an unexpected decomposition property obtained from projecting out the flow variables, considerably simplifying the resulting polyhedral analysis and computations by doing away with metric inequalities, an attendant feature of most successful algorithms on NLP. Under the hose model of feasible demands, we study the polyhedral aspects of NLP, used as the basis of an efficient branch-and-cut algorithm supported by a simple heuristic for generating upper bounds. We provide the results of extensive computational experiments on well-known network design instances.Altın, AyşegülPh.D

Global Optimisation for Energy System

The goal of global optimisation is to find globally optimal solutions, avoiding local optima and other stationary points. The aim of this thesis is to provide more efficient global optimisation tools for energy systems planning and operation. Due to the ongoing increasing of complexity and decentralisation of power systems, the use of advanced mathematical techniques that produce reliable solutions becomes necessary. The task of developing such methods is complicated by the fact that most energy-related problems are nonconvex due to the nonlinear Alternating Current Power Flow equations and the existence of discrete elements. In some cases, the computational challenges arising from the presence of non-convexities can be tackled by relaxing the definition of convexity and identifying classes of problems that can be solved to global optimality by polynomial time algorithms. One such property is known as invexity and is defined by every stationary point of a problem being a global optimum. This thesis investigates how the relation between the objective function and the structure of the feasible set is connected to invexity and presents necessary conditions for invexity in the general case and necessary and sufficient conditions for problems with two degrees of freedom. However, nonconvex problems often do not possess any provable convenient properties, and specialised methods are necessary for providing global optimality guarantees. A widely used technique is solving convex relaxations in order to find a bound on the optimal solution. Semidefinite Programming relaxations can provide good quality bounds, but they suffer from a lack of scalability. We tackle this issue by proposing an algorithm that combines decomposition and linearisation approaches. In addition to continuous non-convexities, many problems in Energy Systems model discrete decisions and are expressed as mixed-integer nonlinear programs (MINLPs). The formulation of a MINLP is of significant importance since it affects the quality of dual bounds. In this thesis we investigate algebraic characterisations of on/off constraints and develop a strengthened version of the Quadratic Convex relaxation of the Optimal Transmission Switching problem. All presented methods were implemented in mathematical modelling and optimisation frameworks PowerTools and Gravity