Hybrid Approaches for MRF Optimization: Combination of Stochastic and Deterministic Methods

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2014. 2. 이경무.Markov Random Field (MRF) models are of fundamental importance in computer vision. Many vision problems have been successfully formulated in MRF optimization. They include stereo matching, segmentation, denoising, and inpainting, to mention just a few. To solve them effectively, numerous algorithms have been developed. Although many of them produce good results for relatively easy problems, they are still unsatisfactory when it comes to more difficult MRF problems such as non-submodular energy functions, strongly coupled MRFs, and high-order clique potentials. In this dissertation, several optimization methods are proposed. The main idea of proposed methods is to combine stochastic and deterministic optimization methods. Stochastic methods encourage more exploration in the solution space. On the other hand, deterministic methods enable more efficient exploitation. By combining those two approaches, it is able to obtain better solution. To this end, two stochastic methodologies are exploited for the framework of combination: Markov chain Monte Carlo (MCMC) and stochastic approximation. First methodology is the MCMC. Based on MCMC framework, population based MCMC (Pop-MCMC), MCMC with General Deterministic algorithms (MCMC-GD), and fusion move driven MCMC (MCMC-F) are proposed. Although MCMC provides an elegant framework of which global convergence is provable, it has the slow convergence rate. To overcome, population-based framework and combination with deterministic methods are used. It thereby enables global moves by exchanging information between samples, which in turn, leads to faster mixing rate. In the view of optimization, it means that we can reach a lower energy state rapidly. Second methodology is the stochastic approximation. In stochastic approximation, the objective function for optimization is approximated in stochastic way. To apply this approach to MRF optimization, graph approximation scheme is proposed for the approximation of the energy function. By using this scheme, it alleviates the problem of non-submodularity and partial labeling. This stochastic approach framework is combined with graph cuts which is very efficient algorithm for easy MRF optimizations. By this combination, fusion with graph approximation-based proposals (GA-fusion) is developed. Extensive experiments support that the proposed algorithms are effective across different classes of energy functions. The proposed algorithms are applied in many different computer vision applications including stereo matching, photo montage, inpaining, image deconvolution, and texture restoration. Those algorithms are further analyzed on synthetic MRF problems while varying the difficulties of the problems as well as the parameters for each algorithm.1 Introduction 1 1.1 Markov random eld . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 MRF and Gibbs distribution . . . . . . . . . . . . . . . . . . 1 1.1.2 MAP estimation and energy minimization . . . . . . . . . . . 2 1.1.3 MRF formulation for computer vision problems . . . . . . . . 3 1.2 Optimizing energy function . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Markov chain Monte Carlo . . . . . . . . . . . . . . . . . . . 7 1.2.2 Stochastic approximation . . . . . . . . . . . . . . . . . . . . 8 1.3 combination of stochastic and deterministic methods . . . . . . . . . 9 1.4 Outline of dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Population-based MCMC 13 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.1 Swendsen-Wang Cuts . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Population-based MCMC . . . . . . . . . . . . . . . . . . . . 19 2.3 Proposed Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.4.1 Segment-based stereo matching . . . . . . . . . . . . . . . . . 31 2.4.2 Parameter analysis . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3 MCMC Combined with General Deterministic Methods 47 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3.1 Population-based sampling framework for MCMC-GD . . . . 53 3.3.2 Kernel design . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.4.1 Analysis on synthetic MRF problems . . . . . . . . . . . . . . 60 3.4.2 Results on real problems . . . . . . . . . . . . . . . . . . . . . 75 3.4.3 Alternative approach: parallel anchor generation . . . . . . . 78 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4 Fusion Move Driven MCMC 89 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.2 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.2.1 Sampling-based optimization . . . . . . . . . . . . . . . . . . 91 4.2.2 MCMC combined with fusion move . . . . . . . . . . . . . . . 92 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5 Fusion with Graph Approximation 101 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.2.1 Graph cuts-based move-making algorithm . . . . . . . . . . . 104 5.2.2 Proposals for fusion approach . . . . . . . . . . . . . . . . . . 106 5.3 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.3.1 Stochastic approximation . . . . . . . . . . . . . . . . . . . . 107 5.3.2 Graph approximation . . . . . . . . . . . . . . . . . . . . . . 108 5.3.3 Overall algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.3.4 Characteristics of approximated function . . . . . . . . . . . 110 5.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 5.4.1 Image deconvolution . . . . . . . . . . . . . . . . . . . . . . . 113 5.4.2 Binary texture restoration . . . . . . . . . . . . . . . . . . . . 115 5.4.3 Analysis on synthetic problems . . . . . . . . . . . . . . . . . 118 5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6 Conclusion 127 6.1 Summary and contribution of the dissertation . . . . . . . . . . . . . 127 6.2 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.2.1 MCMC without detailed balance . . . . . . . . . . . . . . . . 128 6.2.2 Stochastic approximation for higher-order MRF model . . . . 130 Bibliography 131 국문초록 141Docto

Complexity of Discrete Energy Minimization Problems

Discrete energy minimization is widely-used in computer vision and machine learning for problems such as MAP inference in graphical models. The problem, in general, is notoriously intractable, and finding the global optimal solution is known to be NP-hard. However, is it possible to approximate this problem with a reasonable ratio bound on the solution quality in polynomial time? We show in this paper that the answer is no. Specifically, we show that general energy minimization, even in the 2-label pairwise case, and planar energy minimization with three or more labels are exp-APX-complete. This finding rules out the existence of any approximation algorithm with a sub-exponential approximation ratio in the input size for these two problems, including constant factor approximations. Moreover, we collect and review the computational complexity of several subclass problems and arrange them on a complexity scale consisting of three major complexity classes -- PO, APX, and exp-APX, corresponding to problems that are solvable, approximable, and inapproximable in polynomial time. Problems in the first two complexity classes can serve as alternative tractable formulations to the inapproximable ones. This paper can help vision researchers to select an appropriate model for an application or guide them in designing new algorithms.Comment: ECCV'16 accepte

Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems

Optimization methods are at the core of many problems in signal/image processing, computer vision, and machine learning. For a long time, it has been recognized that looking at the dual of an optimization problem may drastically simplify its solution. Deriving efficient strategies which jointly brings into play the primal and the dual problems is however a more recent idea which has generated many important new contributions in the last years. These novel developments are grounded on recent advances in convex analysis, discrete optimization, parallel processing, and non-smooth optimization with emphasis on sparsity issues. In this paper, we aim at presenting the principles of primal-dual approaches, while giving an overview of numerical methods which have been proposed in different contexts. We show the benefits which can be drawn from primal-dual algorithms both for solving large-scale convex optimization problems and discrete ones, and we provide various application examples to illustrate their usefulness

Cycle-based Cluster Variational Method for Direct and Inverse Inference

We elaborate on the idea that loop corrections to belief propagation could be dealt with in a systematic way on pairwise Markov random fields, by using the elements of a cycle basis to define region in a generalized belief propagation setting. The region graph is specified in such a way as to avoid dual loops as much as possible, by discarding redundant Lagrange multipliers, in order to facilitate the convergence, while avoiding instabilities associated to minimal factor graph construction. We end up with a two-level algorithm, where a belief propagation algorithm is run alternatively at the level of each cycle and at the inter-region level. The inverse problem of finding the couplings of a Markov random field from empirical covariances can be addressed region wise. It turns out that this can be done efficiently in particular in the Ising context, where fixed point equations can be derived along with a one-parameter log likelihood function to minimize. Numerical experiments confirm the effectiveness of these considerations both for the direct and inverse MRF inference.Comment: 47 pages, 16 figure