Generalizations of the Multicut Problem for Computer Vision

Graph decomposition has always been a very important concept in machine learning and computer vision. Many tasks like image and mesh segmentation, community detection in social networks, as well as object tracking and human pose estimation can be formulated as a graph decomposition problem. The multicut problem in particular is a popular model to optimize for a decomposition of a given graph. Its main advantage is that no prior knowledge about the number of components or their sizes is required. However, it has several limitations, which we address in this thesis: Firstly, the multicut problem allows to specify only cost or reward for putting two direct neighbours into distinct components. This limits the expressibility of the cost function. We introduce special edges into the graph that allow to define cost or reward for putting any two vertices into distinct components, while preserving the original set of feasible solutions. We show that this considerably improves the quality of image and mesh segmentations. Second, multicut is notorious to be NP-hard for general graphs, that limits its applications to small super-pixel graphs. We define and implement two primal feasible heuristics to solve the problem. They do not provide any guarantees on the runtime or quality of solutions, but in practice show good convergence behaviour. We perform an extensive comparison on multiple graphs of different sizes and properties. Third, we extend the multicut framework by introducing node labels, so that we can jointly optimize for graph decomposition and nodes classification by means of exactly the same optimization algorithm, thus eliminating the need to hand-tune optimizers for a particular task. To prove its universality we applied it to diverse computer vision tasks, including human pose estimation, multiple object tracking, and instance-aware semantic segmentation. We show that we can improve the results over the prior art using exactly the same data as in the original works. Finally, we use employ multicuts in two applications: 1) a client-server tool for interactive video segmentation: After the pre-processing of the video a user draws strokes on several frames and a time-coherent segmentation of the entire video is performed on-the-fly. 2) we formulate a method for simultaneous segmentation and tracking of living cells in microscopy data. This task is challenging as cells split and our algorithm accounts for this, creating parental hierarchies. We also present results on multiple model fitting. We find models in data heavily corrupted by noise by finding components defining these models using higher order multicuts. We introduce an interesting extension that allows our optimization to pick better hyperparameters for each discovered model. In summary, this thesis extends the multicut problem in different directions, proposes algorithms for optimization, and applies it to novel data and settings.Die Zerlegung von Graphen ist ein sehr wichtiges Konzept im maschinellen Lernen und maschinellen Sehen. Viele Aufgaben wie Bild- und Gittersegmentierung, Kommunitätserkennung in sozialen Netzwerken, sowie Objektverfolgung und Schätzung von menschlichen Posen können als Graphzerlegungsproblem formuliert werden. Der Mehrfachschnitt-Ansatz ist ein populäres Mittel um über die Zerlegungen eines gegebenen Graphen zu optimieren. Sein größter Vorteil ist, dass kein Vorwissen über die Anzahl an Komponenten und deren Größen benötigt wird. Dennoch hat er mehrere ernsthafte Limitierungen, welche wir in dieser Arbeit behandeln: Erstens erlaubt der klassische Mehrfachschnitt nur die Spezifikation von Kosten oder Belohnungen für die Trennung von zwei Nachbarn in verschiedene Komponenten. Dies schränkt die Ausdrucksfähigkeit der Kostenfunktion ein und führt zu suboptimalen Ergebnissen. Wir fügen dem Graphen spezielle Kanten hinzu, welche es erlauben, Kosten oder Belohnungen für die Trennung von beliebigen Paaren von Knoten in verschiedene Komponenten zu definieren, ohne die Menge an zulässigen Lösungen zu verändern. Wir zeigen, dass dies die Qualität von Bild- und Gittersegmentierungen deutlich verbessert. Zweitens ist das Mehrfachschnittproblem berüchtigt dafür NP-schwer für allgemeine Graphen zu sein, was die Anwendungen auf kleine superpixel-basierte Graphen einschränkt. Wir definieren und implementieren zwei primal-zulässige Heuristiken um das Problem zu lösen. Diese geben keine Garantien bezüglich der Laufzeit oder der Qualität der Lösungen, zeigen in der Praxis jedoch gutes Konvergenzverhalten. Wir führen einen ausführlichen Vergleich auf vielen Graphen verschiedener Größen und Eigenschaften durch. Drittens erweitern wir den Mehrfachschnitt-Ansatz um Knoten-Kennzeichnungen, sodass wir gemeinsam über Zerlegungen und Knoten-Klassifikationen mit dem gleichen Optimierungs-Algorithmus optimieren können. Dadurch wird der Bedarf der Feinabstimmung einzelner aufgabenspezifischer Löser aus dem Weg geräumt. Um die Allgemeingültigkeit dieses Ansatzes zu überprüfen, haben wir ihn auf verschiedenen Aufgaben des maschinellen Sehens, einschließlich menschliche Posenschätzung, Mehrobjektverfolgung und instanz-bewusste semantische Segmentierung, angewandt. Wir zeigen, dass wir Resultate von vorherigen Arbeiten mit exakt den gleichen Daten verbessern können. Abschließend benutzen wir Mehrfachschnitte in zwei Anwendungen: 1) Ein Nutzer-Server-Werkzeug für interaktive Video Segmentierung: Nach der Vorbearbeitung eines Videos zeichnet der Nutzer Striche auf mehrere Einzelbilder und eine zeit-kohärente Segmentierung des gesamten Videos wird in Echtzeit berechnet. 2) Wir formulieren eine Methode für simultane Segmentierung und Verfolgung von lebenden Zellen in Mikroskopie-Aufnahmen. Diese Aufgabe ist anspruchsvoll, da Zellen sich aufteilen und unser Algorithmus dies in der Erstellung von Eltern-Hierarchien mitberücksichtigen muss. Wir präsentieren außerdem Resultate zur Mehrmodellanpassung. Wir berechnen Modelle in stark verrauschten Daten indem wir mithilfe von Mehrfachschnitten höherer Ordnung Komponenten finden, die diesen Modellen entsprechen. Wir führen eine interessante Erweiterung ein, die es unserer Optimierung erlaubt, bessere Hyperparameter für jedes entdeckte Modell auszuwählen. Zusammenfassend erweitert diese Arbeit den Mehrfachschnitt-Ansatz in unterschiedlichen Richtungen, schlägt Algorithmen zur Inferenz in den resultierenden Modellen vor und wendet ihn auf neuartigen Daten und Umgebungen an

Towards an Aesthetics of the (in)formel: Time, Space and the Dialectical Image in the Music of Varèse, Feldman and Xenakis

This thesis addresses the issue of the modernist musical artwork, specifically in terms of the spatialization of musical time, in aesthetic and music-analytic terms. Firstly, it focuses on the notion of musique informelle as it was expounded in Adorno’s essay ‘Vers une musique informelle,’ (1961) and its place in Gianmario Borio’s elaboration of this in terms of an aesthetics of the informel. Secondly, it proposes a further expansion of these aesthetics via a double strategy: a comparative reading of Walter Benjamin’s critique of philosophies of time (including the work of Henri Bergson), language and objects, and furthermore a reconceptualization of both Adorno’s and Borio’s aesthetics in terms of a new theory of the object (as sound-object) in light of a new reading strategy. This reading is based on Walter Benjamin’s notion of the dialectical image, which proposes a new form of philosophical interpretation. The theorizations of the sound-object and the dialectical image furnish a basis for a re-conceptalization of the (in)formel, allowing for the interpretative reading of the music of three composers in particular: Edgard Varèse, Morton Feldman and Iannis Xenakis. Particularly, the study of a number of their works, including Intégrales (Varèse), On Time and the Instrumental Factor and Words and Music (Feldman), and Duel (Xenakis), reveals what Adorno terms their (truth) content, in their mediation of rationalization and intuition. Finally, it is argued that these modernist works can in turn bring new insights into Adorno’s aesthetics of the modernist work of art

Approximation algorithms for distributed and selfish agents

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mathematics, 2005.Includes bibliographical references (p. 157-165).Many real-world systems involve distributed and selfish agents who optimize their own objective function. In these systems, we need to design efficient mechanisms so that system-wide objective is optimized despite agents acting in their own self interest. In this thesis, we develop approximation algorithms and decentralized mechanisms for various combinatorial optimization problems in such systems. First, we investigate the distributed caching and a general set of assignment problems. We develop an almost tight LP-based ... approximation algorithm and a local search ... approximation algorithm for these problems. We also design efficient decentralized mechanisms for these problems and study the convergence of the corresponding games. In the following chapters, we study the speed of convergence to high quality solutions on (random) best-response paths of players. First, we study the average social value on best response paths in basic-utility, market sharing, and cut games. Then, we introduce the sink equilibrium as a new equilibrium concept. We argue that, unlike Nash equilibria, the selfish behavior of players converges to sink equilibria and all strategic games have a sink equilibrium. To illustrate the use of this new concept, we study the social value of sink equilibria in weighted selfish routing (or weighted congestion) games and valid-utility (or submodular-utility) games. In these games, we bound the average social value on random best-response paths for sink equilibria.. Finally, we study cross-monotonic cost sharings and group-strategyproof mechanisms.(cont.) We study the limitations imposed by the cross-monotonicity property on cost-sharing schemes for several combinatorial optimization games including set cover and metric facility location. We develop a novel technique based on the probabilistic method for proving upper bounds on the budget-balance factor of cross-monotonic cost sharing schemes, deriving tight or nearly-tight bounds for these games. At the end, we extend some of these results to group-strategyproof mechanisms.by Vahab S. Mirrokni.Ph.D

LIPIcs, Volume 244, ESA 2022, Complete Volume

LIPIcs, Volume 244, ESA 2022, Complete Volum

Facility Location and Clock Tree Synthesis

The construction of clock trees and repeater trees are major challenges in chip design. Such trees distribute an electrical clock signal from a source to a set of sinks on a chip. On recent designs there can be millions of repeater trees with only a few up to some hundred sinks and several clock trees with up to some hundred thousand of sinks. In repeater trees the signal has to arrive at each sink not later than an individual required arrival time, while in clock trees it has to arrive at each sink within an individual required arrival time window. In this thesis, we present new theory and algorithms for the construction of clock trees and repeater trees and an essential sub-problem, the Sink Clustering Problem. We also describe our clock tree construction tool BonnClock, which has been used by IBM Microelectronics for the design of hundreds of most complex chips. First, we introduce the Sink Clustering Problem, the main sub-problem of clock tree design. Given a metric space (V,c), a finite set D of terminals with positions p(v) ∈ V and demands d(v) ∈ R ≥ 0 for all v ∈ D, a facility opening cost f ∈ R>0 and a load limit u ∈ R>0 , the task is to find a partition D=D1 ∪ ... ∪ Dk of D and, for all 1 ≤ i ≤ k, a Steiner tree Si for {p(v)| v ∈ Di }. Each cluster (Di ,Si ), 1 ≤ i ≤ k, has to keep the load limit, that means ∑e ∈ E(Si) c(e) +∑s ∈ Di d(s) ≤ u. The goal is to minimize the weighted sum of the length of all Steiner trees plus the number of clusters, i.e. minimize ∑i=1,...,k (∑e ∈ E(Si ) c(e)) +kf. We present the first constant-factor approximation algorithm for the Sink Clustering Problem. It is based on decomposing a minimum spanning tree on the sinks and has an approximation guarantee of 1+2α, where α is the Steiner ratio of the underlying metric. Moreover, we introduce two variants of the algorithm that rely on decomposing an approximate minimum Steiner tree and an approximate minimum traveling salesman tour. These algorithms have approximation guarantees of 3β and 3γ, respectively, where β and γ are the approximation guarantees of the Steiner tree and TSP approximation algorithms, respectively. We also propose two post-optimization algorithms that can further improve an existing clustering. We analyze the structure of the Sink Clustering Problem and exhibit its connections to matroid theory. In particular, we use the property of matroids that for any two bases B1 , B2 there is a bijection p : B1 → B2 so that (B1 \ {b}) ∪ {p(b)} is again a basis for each b ∈ B1. We replace each Steiner tree of an optimum solution by a minimum spanning tree and connect all trees to a new artificial vertex s and get a tree S. In a modified metric the total length of S is a good lower bound for the cost of an optimum solution. Due to the matroid property we can compare a minimum spanning tree T on D ∪ {s} with S; the length of any edge of T is bounded by the length of an edge of S. We introduce the concept of K-dominated functions that helps us to increase the `cost' of certain edges of T while still having the property that the total length of all edges of T ending in a vertex of K ⊆ D is bounded by the total length of all edges of S ending in a vertex of K. Applying this procedure to the sets of a laminar family on D yields an improved lower bound. The bound can be further improved by combining it with a lower bound for the length of a minimum Steiner tree on D. For this bound we prove the following lemma: For any family of trees T = {T1 ,..., Tk } with V(Ti ) ⊂ D, 1 ≤ i ≤ k, with the property that for any subset T' ⊆ T the trees in T' cover at least | T' |+1 vertices, there exists an edge ei ∈ E(Ti ) for i=1,..., k such that these edges E={ei | 1 ≤ i ≤ k} form a forest, i.e. the set does not contain an edge twice and it does not contain a circuit. Our experimental results on real-world instances from clock tree design show that the cost of the solutions computed by our algorithms is in average only 10% over the best lower bound. Moreover, we compare our algorithm to another clustering algorithm used in industry. The results show that the total cost of our solutions is 10% less than the cost of the solutions computed by the competitive tool. Clock trees have to satisfy several timing constraints. More precisely, the signal has to reach each sink within an individual required arrival time window. Sinks can only be clustered together if their required arrival time windows have a point of time in common. Typically, all required arrival time windows are the same. In this case we have the Sink Clustering Problem defined above. However, there are clock trees where the sinks have different required arrival time windows. This motivates a generalization of the Sink Clustering Problem where each sink additionally has an individual time window. As further constraint the time windows of the sinks of a cluster must have at least one point of time in common. We study the Sink Clustering Problem with Time Windows and present a polynomial O(log s)-approximation algorithm for this problem, where s is the size of a minimum clique partition in the interval graph induced by the time windows. Our algorithm is based on a divide and conquer approach and uses the approximation algorithms for the Sink Clustering Problem on sub-sets of the instance. We show that the approximation guarantee of the algorithm is tight. For the practical construction of clock trees we present our algorithm BonnClock. BonnClock builds a clock tree combining a bottom-up clustering and a top-down partitioning strategy. In the bottom-up phase BonnClock is using the Sink Clustering Algorithm in order to determine the drivers of unconnected sinks or inverters. The `global' topology of the tree is determined by the top-down partitioning considering big blockages and timing restrictions. BonnClock uses a dynamic program in order to determine the sizes of the inverters that are inserted. All components of the algorithm are discussed in detail. As part of this thesis, we have also implemented this algorithm. BonnClock has become the standard tool to construct clock trees within IBM. We show experimental results with comparisons to another industrial clock tree construction tool and to lower bounds for the power consumption. It turns out that - mainly due to the Sink Clustering Algorithm - our power consumption is much smaller than with the other tool and only one third over the lower bound. Finally, we consider the repeater tree construction problem. In contrast to clock trees, each sink has a latest required arrival time instead of a time window. We describe a simple algorithm to build such trees where we insert the sinks one by one into an existing tree. Depending on the optimization goal we show a variant of the algorithm computing trees of almost optimal length or trees with guaranteed best possible performance. Moreover, we analyze the topology of trees with best or almost best performance more closely. Such trees are equivalent to minimax and almost minimax trees: Let a1 , ... , an ∈ N ≥ 0 be a set of numbers. The weight of a tree with n leaves is the maximum over all leaves i of the depth of leaf i plus ai. For a non-negative integral constant c the goal is to build a binary tree with weight at most the optimum weight plus c. This problem can be solved optimally by a greedy algorithm. However, we are interested in the online version of this problem where we have to insert the leaf i with weight ai into the tree without knowing n and the following weights aj, j> i. We give necessary and sufficient conditions for an online algorithm to compute trees of weight at most the optimum weight plus c. Moreover, we show how these conditions can be verified efficiently. We obtain an online algorithm that computes an optimum tree in O(nlog n) time. Finally, we study a further mathematical model of repeater trees that considers that additional delay caused by a bifurcation of a tree can be distributed partially to the two branches. For c∈ R>0 and a set L ⊆ {(l1 ,l2 ) ∈ R2 ≥ 0 | l1 +l2 = c} of two-element sets of non-negative real numbers we consider rooted binary trees with the property that the two edges emanating from every non-leaf are assigned lengths l1 and l2 with { l1 ,l2 } ? L. We study the asymptotic growth of the maximum number of leaves of bounded depths in such trees and the existence of such trees with leaves at individually specified maximum depths. Our results yield better lower bounds for repeater trees

Signalisierte Netzwerkflüsse - Optimierung von Lichtsignalanlagen und Vorwegweisern und daraus resultierende Netzwerkflussprobleme

Guideposts and traffic signals are important devices for controlling inner-city traffic and their optimized operation is essential for efficient traffic flow without congestion. In this thesis, we develop a mathematical model for guideposts and traffic signals in the context of network flow theory. Guideposts lead to confluent flows where each node in the network may have at most one outgoing flow-carrying arc. The complexity of finding maximum confluent flows is studied and several polynomial time algorithms for special graph classes are developed. For traffic signal optimization, a cyclically time-expanded model is suggested which provides the possibility of the simultaneous optimization of offsets and traffic assignment. Thus, the influence of offsets on travel times can be accounted directly. The potential of the presented approach is demonstrated by simulation of real-world instances.Vorwegweiser und Lichtsignalanlagen sind wichtige Elemente zur Steuerung innerstädtischen Verkehrs und ihre optimale Nutzung ist von entscheidender Bedeutung für einen staufreien Verkehrsfluss. In dieser Arbeit werden Vorwegweiser und Lichtsignalanlagen mittels der Netzwerkflusstheorie mathematisch modelliert. Vorwegweiser führen dabei zu konfluenten Flüssen, bei denen Fluss einen Knoten des Netzwerks nur gebündelt auf einer einzigen Kante verlassen darf. Diese konfluenten Flüsse werden hinsichtlich ihrer Komplexität untersucht und es werden Polynomialzeitalgorithmen für das Finden maximaler Flüsse auf ausgewählten Graphenklassen vorgestellt. Für die Versatzzeitoptimierung von Lichtsignalanlagen wird ein zyklisch zeitexpandiertes Modell entwickelt, das die gleichzeitige Optimierung der Verkehrsumlegung ermöglicht. So kann der Einfluss geänderter Versatzzeiten auf die Fahrzeiten direkt berücksichtigt werden. Die Leistungsfähigkeit dieses Ansatzes wird mit Hilfe von Simulationen realistischer Szenarien nachgewiesen

From chemical gardens to chemobrionics

Chemical gardens in laboratory chemistries ranging from silicates to polyoxometalates, in applications ranging from corrosion products to the hydration of Portland cement, and in natural settings ranging from hydrothermal vents in the ocean depths to brinicles beneath sea ice. In many chemical-garden experiments, the structure forms as a solid seed of a soluble ionic compound dissolves in a solution containing another reactive ion. In general any alkali silicate solution can be used due to their high solubility at high pH. The cation should not precipitate with the counterion of the metal salt used as seed. A main property of seed chemical-garden experiments is that initially, when the fluid is not moving under buoyancy or osmosis, the delivery of the inner reactant is diffusion controlled. Another experimental technique that isolates one aspect of chemical-garden formation is to produce precipitation membranes between different aqueous solutions by introducing the two solutions on either side of an inert carrier matrix. Chemical gardens may be grown upon injection of solutions into a so-called Hele-Shaw cell, a quasi-two-dimensional reactor consisting in two parallel plates separated by a small gap

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field