A Message Passing Algorithm for the Minimum Cost Multicut Problem

We propose a dual decomposition and linear program relaxation of the NP -hard minimum cost multicut problem. Unlike other polyhedral relaxations of the multicut polytope, it is amenable to efficient optimization by message passing. Like other polyhedral elaxations, it can be tightened efficiently by cutting planes. We define an algorithm that alternates between message passing and efficient separation of cycle- and odd-wheel inequalities. This algorithm is more efficient than state-of-the-art algorithms based on linear programming, including algorithms written in the framework of leading commercial software, as we show in experiments with large instances of the problem from applications in computer vision, biomedical image analysis and data mining.Comment: Added acknowledgment

FastDOG: Fast Discrete Optimization on GPU

We present a massively parallel Lagrange decomposition method for solving 0--1 integer linear programs occurring in structured prediction. We propose a new iterative update scheme for solving the Lagrangean dual and a perturbation technique for decoding primal solutions. For representing subproblems we follow Lange et al. (2021) and use binary decision diagrams (BDDs). Our primal and dual algorithms require little synchronization between subproblems and optimization over BDDs needs only elementary operations without complicated control flow. This allows us to exploit the parallelism offered by GPUs for all components of our method. We present experimental results on combinatorial problems from MAP inference for Markov Random Fields, quadratic assignment and cell tracking for developmental biology. Our highly parallel GPU implementation improves upon the running times of the algorithms from Lange et al. (2021) by up to an order of magnitude. In particular, we come close to or outperform some state-of-the-art specialized heuristics while being problem agnostic. Our implementation is available at https://github.com/LPMP/BDD.Comment: Published at CVPR 2022. Alert before printing: last 10 pages just contains detailed results tabl

Discrete graphical models -- an optimization perspective

This monograph is about discrete energy minimization for discrete graphical models. It considers graphical models, or, more precisely, maximum a posteriori inference for graphical models, purely as a combinatorial optimization problem. Modeling, applications, probabilistic interpretations and many other aspects are either ignored here or find their place in examples and remarks only. It covers the integer linear programming formulation of the problem as well as its linear programming, Lagrange and Lagrange decomposition-based relaxations. In particular, it provides a detailed analysis of the polynomially solvable acyclic and submodular problems, along with the corresponding exact optimization methods. Major approximate methods, such as message passing and graph cut techniques are also described and analyzed comprehensively. The monograph can be useful for undergraduate and graduate students studying optimization or graphical models, as well as for experts in optimization who want to have a look into graphical models. To make the monograph suitable for both categories of readers we explicitly separate the mathematical optimization background chapters from those specific to graphical models.Comment: 270 page

A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems

International audienceSzeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved changed significantly. Specifically , the models today often include higher order interactions, flexible connectivity structures, large label-spaces of different car-dinalities, or learned energy tables. To reflect these changes, we provide a modernized and enlarged study. We present an empirical comparison of more than 27 state-of-the-art optimization techniques on a corpus of 2,453 energy minimization instances from diverse applications in computer vision. To ensure reproducibility, we evaluate all methods in the OpenGM 2 framework and report extensive results regarding runtime and solution quality. Key insights from our study agree with the results of Szeliski et al. for the types of models they studied. However, on new and challenging types of models our findings disagree and suggest that polyhedral methods and integer programming solvers are competitive in terms of runtime and solution quality over a large range of model types

Combinatorial optimization under ellipsoidal uncertainty

We study combinatorial problems with ellipsoidal uncertainty in the objective function concerning their theoretical and practical solvability. Ellipsoidal uncertainty is a natural model when the coefficients are normally distributed random variables. Robust versions of typical combinatorial problems can be very hard to solve compared to their linear versions. Complexity and approaches differ fundamentally depending on whether uncorrelated or correlated uncertainty occurs. We distinguish between these two cases and consider first the unconstrained binary optimization under uncorrelated ellipsoidal uncertainty. For this we develop an algorithm which computes an optimal solution by merely sorting the variables and, correspondingly, has a running time of O(n log n). The algorithm is based on the diminishing returns-property, which is characteristic for submodular functions. We introduce a new and a more general p-norm-uncertainty and show that with only slight modifications the sorting algorithm can be easily applied. We also extend the algorithm to general integer variables, which in this case only leads to a pseudo-polynomial time. The next step to the general case is investigation of problems with arbitrary combinatorial sets X ⊆ {0, 1}n under uncorrelated ellipsoidal uncertainty. For this case we embed the O(n log n)-algorithm for the unconstrained binary problems into a Lagrangean decomposition approach. The approach separates the objective function from the combinatorial structure applying Lagrangean relaxation to some artificial connecting constraints. This creates two subproblems, one of which is the linear version of the combinatorial problem and the other one is just the unconstrained binary uncorrelated problem, which can be solved using the O(n log n)-algorithm. The solutions of the subproblems are used to obtain primal and dual bounds which are used in a branch and bound-approach. The approach shows an excellent performance in practice. In the correlated case already the unconstrained binary problem turns out to be strongly NP-hard. Here we also define a branch and bound-approach, now with lower bounds determined by underestimation of the given ellipsoid with certainly defined axis-parallel ellipsoids. We use this idea to extend the decomposition approach to general combinatorial problems under correlated uncertainty. In contrast to the uncorrelated case the uncertain subproblem of the decomposition is here strongly NP-hard in itself. We solve it approximately using the developed underestimators which are determined in a preprocessing step. The approach offers room for improvement concerning in the primal extent a faster computation of the underestimators, which is done by solving semidefinite programs

Lifted edges as connectivity priors for multicut and disjoint paths

This work studies graph decompositions and their representation by 0/1 labeling of edges. We study two problems. The first is multicut (MC) which represents decompositions of undirected graphs (clustering of nodes into connected components). The second is disjoint paths (DP) in directed acyclic graphs where the clusters correspond to node- disjoint paths. Unlike an alternative representation by node labeling, the number of clusters is not part of the input but is fully determined by the costs of edges. Our main interest is to study connectivity priors represented by so-called lifted edges in the two problems. The cost of a lifted edge expresses whether its endpoints should belong to the same cluster (path) in the optimal decomposition. We call the resulting problems lifted multicut (LMC) and lifted disjoint paths (LDP). The extension of MC to LMC was originally motivated by image segmentation where the information about the connectivity between non-neighboring pixels or superpixels led to a significant quality improvement. After that, LMC was successfully applied to other problems like multiple object tracking (MOT) which is also the main application of our proposed LDP model. Our study of lifted multicut concentrates on partial LMC represented by labeling of a subset of (lifted) edges. Given partial labeling, we conclude that deciding whether a complete LMC consistent with the partial labels exists is NP-complete. Similarly, we conclude that deciding whether an unlabeled edge exists such that its label is determined by the labels of other edges is NP-hard. After that, we present metrics for comparing (partial) graph decompositions. Finally, we study the properties of the LMC polytope. The largest part of this work is dedicated to the proposed LDP problem. We prove that this problem is NP-hard and propose an optimal integer linear programming (ILP) solver. In order to enable its global optimization, we formulate several classes of linear inequalities that produce a high-quality LP relaxation. Additionally, we propose efficient cutting plane algorithms for separating the proposed linear inequalities. Despite the advanced constraints and efficient separation routines, the general time complexity of our optimal ILP solver remains exponential. In order to solve even larger instances, we introduce an approximate LDP solver based on Lagrange decomposition. LDP is a convenient model for MOT because the underlying disjoint paths model naturally leads to trajectories of objects. Moreover, lifted edges encode long-range temporal interactions and thus help to prevent id switches and re-identify persons. Our tracker using the optimal LDP solver achieves nearly optimal assignments w.r.t. input detections. Consequently, it was a leading tracker on three benchmarks of the MOT challenge MOT15/16/17, improving significantly over state-of-the-art at the time of its publication. Our approximate LDP solver enables us to process the MOT15/16/17 benchmarks without sacrificing solution quality and allows for solving large and dense instances of a challenging dataset MOT20. On all these four standard MOT benchmarks we achieved performance comparable or better than state-of-the-art methods (at the time of publication) including our tracker based on the optimal LDP solver.Diese Arbeit studiert Graphenzerlegungen und ihre Repräsentation durch 0/1-wertige Kantenbelegungen. Das erste Problem ist das Mehrfachschnittproblem. Es repräsentiert Zerlegungen von ungerichteten Graphen (Cluster von Knoten sodass jeder Cluster eine Zusammenhangskomponente repräsentiert). Das zweite Problem ist die Suche von disjunkten Pfaden in einem gerichteten azyklischen Graph in dem die Cluster knotendisjunkten Pfaden entsprechen. Im Unterschied zu der alternativen Repräsentation durch Knotenbelegungen ist die Zahl von Clustern nicht im Voraus gegeben, sondern sie ist abhängig von den Kosten der Kanten. Der Fokus dieser Arbeit ist die Erforschung von hochgezogenen Kannten, die eine apriori Information über Verbundenheit von Knoten in Clustern respektive durch Pfade in den zwei Problemen darstellen. Die Kosten einer hochgezogenen Kante drücken aus, ob ihre Knoten zu dem gleichen Cluster (Pfad) in der optimalen Zerlegung gehören sollten. Wir bezeichnen diese neuen Probleme als das hochgezogene Mehrfachschnittproblem und das Problem der hochgezogenen disjunkten Pfade. Die Erweiterung des Mehrfachschnittproblems zu dem hochgezogenen Mehrfachschnittproblem wurde ursprünglich durch die Bildsegmentierung motiviert, für die die Information über Verbundenheit von nicht benachbarten Pixeln oder Superpixeln zu einer bedeutenden Verbesserung der Qualität führte. Danach wurde das hochgezogene Mehrfachschnittproblem zu der Lösung von anderen Problemen wie zum Beispiel der Verfolgung von mehreren Objekten in einem Video angewendet. Diese Aufgabe ist auch die Hauptanwendung des vorgeschlagenen Problems der hochgezogenen disjunkte Pfade. In unserer Untersuchung des hochgezogenen Mehrfachschnittproblems konzentrieren wir uns auf das teilweise hochgezogene Mehrfachschnittproblem. Das Problem wird durch eine Belegung einer Teilmenge der (hochgezogenen) Kanten repräsentiert. Wir beweisen, dass es NP-vollständig ist zu entscheiden, ob ein kompletter hochgezogener Mehrfachschnitt existiert, der einer gegebenen teilweisen Kantenbezeichnung entspricht. In analogerWeise beweisen wir, dass es NP-schwer ist zu entscheiden, ob eine nicht belegte Kante existiert, deren Belegung durch die Belegungen anderer Kanten entschieden ist. Danach präsentieren wir Metriken zum Vergleich von (teilweisen) Graphenzerlegungen. Schließlich untersuchen wir Eigenschaften des hochgezogenen Mehrfachschnitt-Polytops. Der größte Teil dieser Arbeit widmet sich dem von uns vorgeschlagenen Problem der hochgezogenen disjunkten Pfade. Wir beweisen, dass es NP-schwer ist. Wir formulieren es als ein ganzzahliges lineares Optimierungsproblem und implementieren ein Programm für dessen optimale Lösung. Um die globale Optimierung zu ermöglichen, formulieren wir mehrere Klassen von linearen Ungleichungen, die zu einer linearen Relaxierung mit einer hohen Qualität führen. Zusätzlich präsentieren wir ein effektives Schnittebenenverfahren für die Separierung der vorgeschlagenen Ungleichungen. Trotz der fortgeschrittenen Ungleichungen und der Effizienz der Schnittebenenseparierung in unserem optimalen Löser bleibt die allgemeine Komplexität des Algorithmus exponentiell. Um noch kompliziertere Instanzen zu lösen, präsentieren wir einen approximativen Löser, der auf Lagrange-Dualität aufbaut. Hochgezogene disjunkte Pfade sind ein praktisches Modell für die Verfolgung von mehreren Objekten, weil die disjunkten Pfade eine natürliche Repräsentation von Trajektorien der Objekten darstellen. Außerdem repräsentieren die hochgezogenen Kanten Interaktionen einer langen zeitlichen Reichweite. Deswegen helfen sie dieselbe Person in zeitlich weiter auseinander liegenden Zeitpunkten wieder zu identifizieren und Verwechselungen ihrer Identität zu verhindern. Aus diesem Grund war unsere Methode zur Zeit ihrer Publikation die beste für drei Vergleichsdatensätzen MOT Challenge MOT15/16/17 für die Verfolgung von mehreren Objekten. Im Vergleich zu den bisherigen besten Methoden war ihre Leistung sogar bedeutend höher. Unsere approximative Methode für hochgezogene disjunkte Pfade ermöglicht uns die Vergleichsdatensätzen MOT15/16/17 zu verarbeiten ohne die Qualität der Lösungen zu vermindern und erlaubt uns, die großen Instanzen mit hoher Personendichte des anspruchsvolleren Datensatzes MOT20 zu lösen. Zur Zeit ihrer Publikation erreichte die Methode vergleichbare oder bessere Ergebnisse als die bisherigen besten Methoden einschließlich unseres optimalen Löser für hochgezogene disjunkte Pfade