컴퓨터비전을 위한 그래프정합과 고차그래프정합: 새로운 알고리즘과 분석에 관한 연구

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 8. 이경무.Establishing image feature correspondences is fundamental problem in computer vision and machine learning research fields. Myriad of graph matching algorithms have been proposed to tackle this problem by regarding correspondence problem as a graph matching problem. However, the graph matching problem is challenging since there are various types of noises in real world scenarioe.g., non-rigid motion, view-point change, and background clutter. The objective of this dissertation is to propose robust graph matching algorithms for feature correspondence task in computer vision and to investigate an effective graph matching strategy. For the purpose, at first, two robust simulation based graph matching algorithms are introduced: the one is based on Random Walks simulation and the other is based on Markov Chain Monte Carlo sampling simulation. Secondly, two different graph matching formulations and their transformal relation are studied since equivalence between two formulations are not well studied in graph matching fields. It is demonstrated that conventional graph matching algorithms can solve both types of formulations by proposing conversion principle between two formulations. Finally, these whole statements are extended into hypergraph matching problem by introducing two robust hypergraph matching algorithms which are based on Random Walks and Markov Chain Monte Carlo, by relating two different hypergraph matching formulations, and by reinterpreting previous hypergraph matching algorithms into their counterpart formulations. Throughout chapters in this dissertation, comparative and extensive experiments verify characteristics of formulations, transformal relations, and algorithms. Synthetic graph matching problems as well as real image feature correspondence problems are performed in various and severe noise conditions.Chapter 1 Introduction 1 1.1 Graph Matching Problem 1 1.1.1 Graph Matching for Computer Vision 1 1.1.2 Graph Matching Formulation 2 1.1.3 Extension to Hypergraph Matching 5 1.2 Outline of Dissertation 6 Chapter 2 Graph Matching via Random Walks 9 2.1 Introduction 9 2.1.1 Related Works 10 2.2 Problem Formulation 12 2.2.1 Graph Matching Formulation 12 2.2.2 Hypergraphs Matching Formulation 13 2.3 Graph Matching via Random Walks 16 2.3.1 Random Walks for Graph Matching 16 2.3.2 Reweighting Jumps for Graph Matching 19 2.4 Hypergraph Matching via Random Walks 22 2.4.1 Hypergraph Random Walks 22 2.4.2 Reweighting Jumps for Hypergraph Matching 23 2.5 Experiments 26 2.5.1 Random Graph Matching 27 2.5.2 Synthetic Point Matching 34 2.5.3 Image Sequence Matching 37 2.5.4 Image Feature Matching 39 2.6 Conclusion 44 Chapter 3 Graph Matching via Markov Chain Monte Carlo 45 3.1 Introduction 45 3.2 Graph Matching Formulation 47 3.3 Algorithm 49 3.3.1 State Transition 49 3.3.2 Energy Formulation 49 3.3.3 Data-Driven Proposal 51 3.4 Hypergraph Extension 53 3.4.1 Hypergraph Matching Problem 53 3.4.2 Energy Formulation & Data-Driven Proposal 54 3.5 Experiment 54 3.5.1 Random Graph Matching Problem 54 3.5.2 Random Hypergraph Matching Problem 58 3.6 Conclusion 59 Chapter 4 Graph and Hypergraph Matching Revisited 63 4.1 Introduction 63 4.2 Related Works 65 4.3 Two Types of Formulations 66 4.3.1 Adjacency-based Formulation 67 4.3.2 Affinity-based Formulation 69 4.3.3 Relation between Two Formulations 70 4.4 Affinity Measures 72 4.5 Existing Methods & Re-interpretations 74 4.5.1 Spectral Matching 74 4.5.2 Integer Projected Fixed Point 75 4.5.3 Reweighted Random Walks Matching 76 4.5.4 Factorized Graph Matching 77 4.6 High-order Methods & Reinterpretations 78 4.6.1 Hypergraph Matching by Zass and Shashua 81 4.6.2 SVD-based Hypergraph Matching 82 4.6.3 Tensor Power Iteration based Hypergraph Matching 82 4.6.4 Reweighted Random Walks for Hypergraph Matching 83 4.6.5 Discrete Hypergraph Matching 85 4.7 Experiments & Comparison 85 4.8 Conclusion 102 Chapter 5 Conclusion 105 5.1 Summary and Contribution of Dissertation 105 5.2 Future Works 107 Bibliography 109 국문 초록 117Docto

Towards Resistance Sparsifiers

We study resistance sparsification of graphs, in which the goal is to find a sparse subgraph (with reweighted edges) that approximately preserves the effective resistances between every pair of nodes. We show that every dense regular expander admits a $(1+\epsilon)$-resistance sparsifier of size $\tilde O(n/\epsilon)$, and conjecture this bound holds for all graphs on $n$ nodes. In comparison, spectral sparsification is a strictly stronger notion and requires $\Omega(n/\epsilon^2)$ edges even on the complete graph. Our approach leads to the following structural question on graphs: Does every dense regular expander contain a sparse regular expander as a subgraph? Our main technical contribution, which may of independent interest, is a positive answer to this question in a certain setting of parameters. Combining this with a recent result of von Luxburg, Radl, and Hein~(JMLR, 2014) leads to the aforementioned resistance sparsifiers

Cheeger Inequalities for Vertex Expansion and Reweighted Eigenvalues

The classical Cheeger's inequality relates the edge conductance $\phi$ of a graph and the second smallest eigenvalue $\lambda_2$ of the Laplacian matrix. Recently, Olesker-Taylor and Zanetti discovered a Cheeger-type inequality $\psi^2 / \log |V| \lesssim \lambda_2^* \lesssim \psi$ connecting the vertex expansion $\psi$ of a graph $G=(V,E)$ and the maximum reweighted second smallest eigenvalue $\lambda_2^*$ of the Laplacian matrix. In this work, we first improve their result to $\psi^2 / \log d \lesssim \lambda_2^* \lesssim \psi$ where $d$ is the maximum degree in $G$, which is optimal assuming the small-set expansion conjecture. Also, the improved result holds for weighted vertex expansion, answering an open question by Olesker-Taylor and Zanetti. Building on this connection, we then develop a new spectral theory for vertex expansion. We discover that several interesting generalizations of Cheeger inequalities relating edge conductances and eigenvalues have a close analog in relating vertex expansions and reweighted eigenvalues. These include an analog of Trevisan's result on bipartiteness, an analog of higher order Cheeger's inequality, and an analog of improved Cheeger's inequality. Finally, inspired by this connection, we present negative evidence to the $0/1$-polytope edge expansion conjecture by Mihail and Vazirani. We construct $0/1$-polytopes whose graphs have very poor vertex expansion. This implies that the fastest mixing time to the uniform distribution on the vertices of these $0/1$-polytopes is almost linear in the graph size. This does not provide a counterexample to the conjecture, but this is in contrast with known positive results which proved poly-logarithmic mixing time to the uniform distribution on the vertices of subclasses of $0/1$-polytopes.Comment: 65 pages, 1 figure. Minor change

Product graph-based higher order contextual similarities for inexact subgraph matching

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record Many algorithms formulate graph matching as an optimization of an objective function of pairwise quantification of nodes and edges of two graphs to be matched. Pairwise measurements usually consider local attributes but disregard contextual information involved in graph structures. We address this issue by proposing contextual similarities between pairs of nodes. This is done by considering the tensor product graph (TPG) of two graphs to be matched, where each node is an ordered pair of nodes of the operand graphs. Contextual similarities between a pair of nodes are computed by accumulating weighted walks (normalized pairwise similarities) terminating at the corresponding paired node in TPG. Once the contextual similarities are obtained, we formulate subgraph matching as a node and edge selection problem in TPG. We use contextual similarities to construct an objective function and optimize it with a linear programming approach. Since random walk formulation through TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities and better discrimination among the nodes and edges. Experimental results shown on synthetic as well as real benchmarks illustrate that higher order contextual similarities increase discriminating power and allow one to find approximate solutions to the subgraph matching problem.European Union Horizon 202

Learning Combinatorial Embedding Networks for Deep Graph Matching

Graph matching refers to finding node correspondence between graphs, such that the corresponding node and edge's affinity can be maximized. In addition with its NP-completeness nature, another important challenge is effective modeling of the node-wise and structure-wise affinity across graphs and the resulting objective, to guide the matching procedure effectively finding the true matching against noises. To this end, this paper devises an end-to-end differentiable deep network pipeline to learn the affinity for graph matching. It involves a supervised permutation loss regarding with node correspondence to capture the combinatorial nature for graph matching. Meanwhile deep graph embedding models are adopted to parameterize both intra-graph and cross-graph affinity functions, instead of the traditional shallow and simple parametric forms e.g. a Gaussian kernel. The embedding can also effectively capture the higher-order structure beyond second-order edges. The permutation loss model is agnostic to the number of nodes, and the embedding model is shared among nodes such that the network allows for varying numbers of nodes in graphs for training and inference. Moreover, our network is class-agnostic with some generalization capability across different categories. All these features are welcomed for real-world applications. Experiments show its superiority against state-of-the-art graph matching learning methods.Comment: ICCV2019 oral. Code available at https://github.com/Thinklab-SJTU/PCA-G