Hybrid Approaches for MRF Optimization: Combination of Stochastic and Deterministic Methods

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2014. 2. 이경무.Markov Random Field (MRF) models are of fundamental importance in computer vision. Many vision problems have been successfully formulated in MRF optimization. They include stereo matching, segmentation, denoising, and inpainting, to mention just a few. To solve them effectively, numerous algorithms have been developed. Although many of them produce good results for relatively easy problems, they are still unsatisfactory when it comes to more difficult MRF problems such as non-submodular energy functions, strongly coupled MRFs, and high-order clique potentials. In this dissertation, several optimization methods are proposed. The main idea of proposed methods is to combine stochastic and deterministic optimization methods. Stochastic methods encourage more exploration in the solution space. On the other hand, deterministic methods enable more efficient exploitation. By combining those two approaches, it is able to obtain better solution. To this end, two stochastic methodologies are exploited for the framework of combination: Markov chain Monte Carlo (MCMC) and stochastic approximation. First methodology is the MCMC. Based on MCMC framework, population based MCMC (Pop-MCMC), MCMC with General Deterministic algorithms (MCMC-GD), and fusion move driven MCMC (MCMC-F) are proposed. Although MCMC provides an elegant framework of which global convergence is provable, it has the slow convergence rate. To overcome, population-based framework and combination with deterministic methods are used. It thereby enables global moves by exchanging information between samples, which in turn, leads to faster mixing rate. In the view of optimization, it means that we can reach a lower energy state rapidly. Second methodology is the stochastic approximation. In stochastic approximation, the objective function for optimization is approximated in stochastic way. To apply this approach to MRF optimization, graph approximation scheme is proposed for the approximation of the energy function. By using this scheme, it alleviates the problem of non-submodularity and partial labeling. This stochastic approach framework is combined with graph cuts which is very efficient algorithm for easy MRF optimizations. By this combination, fusion with graph approximation-based proposals (GA-fusion) is developed. Extensive experiments support that the proposed algorithms are effective across different classes of energy functions. The proposed algorithms are applied in many different computer vision applications including stereo matching, photo montage, inpaining, image deconvolution, and texture restoration. Those algorithms are further analyzed on synthetic MRF problems while varying the difficulties of the problems as well as the parameters for each algorithm.1 Introduction 1 1.1 Markov random eld . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 MRF and Gibbs distribution . . . . . . . . . . . . . . . . . . 1 1.1.2 MAP estimation and energy minimization . . . . . . . . . . . 2 1.1.3 MRF formulation for computer vision problems . . . . . . . . 3 1.2 Optimizing energy function . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Markov chain Monte Carlo . . . . . . . . . . . . . . . . . . . 7 1.2.2 Stochastic approximation . . . . . . . . . . . . . . . . . . . . 8 1.3 combination of stochastic and deterministic methods . . . . . . . . . 9 1.4 Outline of dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Population-based MCMC 13 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2.1 Swendsen-Wang Cuts . . . . . . . . . . . . . . . . . . . . . . 16 2.2.2 Population-based MCMC . . . . . . . . . . . . . . . . . . . . 19 2.3 Proposed Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.4.1 Segment-based stereo matching . . . . . . . . . . . . . . . . . 31 2.4.2 Parameter analysis . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3 MCMC Combined with General Deterministic Methods 47 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3.1 Population-based sampling framework for MCMC-GD . . . . 53 3.3.2 Kernel design . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.4.1 Analysis on synthetic MRF problems . . . . . . . . . . . . . . 60 3.4.2 Results on real problems . . . . . . . . . . . . . . . . . . . . . 75 3.4.3 Alternative approach: parallel anchor generation . . . . . . . 78 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4 Fusion Move Driven MCMC 89 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.2 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 4.2.1 Sampling-based optimization . . . . . . . . . . . . . . . . . . 91 4.2.2 MCMC combined with fusion move . . . . . . . . . . . . . . . 92 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5 Fusion with Graph Approximation 101 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.2 Related works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.2.1 Graph cuts-based move-making algorithm . . . . . . . . . . . 104 5.2.2 Proposals for fusion approach . . . . . . . . . . . . . . . . . . 106 5.3 Proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.3.1 Stochastic approximation . . . . . . . . . . . . . . . . . . . . 107 5.3.2 Graph approximation . . . . . . . . . . . . . . . . . . . . . . 108 5.3.3 Overall algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.3.4 Characteristics of approximated function . . . . . . . . . . . 110 5.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 5.4.1 Image deconvolution . . . . . . . . . . . . . . . . . . . . . . . 113 5.4.2 Binary texture restoration . . . . . . . . . . . . . . . . . . . . 115 5.4.3 Analysis on synthetic problems . . . . . . . . . . . . . . . . . 118 5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6 Conclusion 127 6.1 Summary and contribution of the dissertation . . . . . . . . . . . . . 127 6.2 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.2.1 MCMC without detailed balance . . . . . . . . . . . . . . . . 128 6.2.2 Stochastic approximation for higher-order MRF model . . . . 130 Bibliography 131 국문초록 141Docto

Higher Order Energies for Image Segmentation

A novel energy minimization method for general higher-order binary energy functions is proposed in this paper. We first relax a discrete higher-order function to a continuous one, and use the Taylor expansion to obtain an approximate lower-order function, which is optimized by the quadratic pseudo-boolean optimization (QPBO) or other discrete optimizers. The minimum solution of this lower-order function is then used as a new local point, where we expand the original higher-order energy function again. Our algorithm does not restrict to any specific form of the higher-order binary function or bring in extra auxiliary variables. For concreteness, we show an application of segmentation with the appearance entropy, which is efficiently solved by our method. Experimental results demonstrate that our method outperforms state-of-the-art methods

Advances in Graph-Cut Optimization: Multi-Surface Models, Label Costs, and Hierarchical Costs

Computer vision is full of problems that are elegantly expressed in terms of mathematical optimization, or energy minimization. This is particularly true of low-level inference problems such as cleaning up noisy signals, clustering and classifying data, or estimating 3D points from images. Energies let us state each problem as a clear, precise objective function. Minimizing the correct energy would, hypothetically, yield a good solution to the corresponding problem. Unfortunately, even for low-level problems we are confronted by energies that are computationally hard—often NP-hard—to minimize. As a consequence, a rather large portion of computer vision research is dedicated to proposing better energies and better algorithms for energies. This dissertation presents work along the same line, specifically new energies and algorithms based on graph cuts. We present three distinct contributions. First we consider biomedical segmentation where the object of interest comprises multiple distinct regions of uncertain shape (e.g. blood vessels, airways, bone tissue). We show that this common yet difficult scenario can be modeled as an energy over multiple interacting surfaces, and can be globally optimized by a single graph cut. Second, we introduce multi-label energies with label costs and provide algorithms to minimize them. We show how label costs are useful for clustering and robust estimation problems in vision. Third, we characterize a class of energies with hierarchical costs and propose a novel hierarchical fusion algorithm with improved approximation guarantees. Hierarchical costs are natural for modeling an array of difficult problems, e.g. segmentation with hierarchical context, simultaneous estimation of motions and homographies, or detecting hierarchies of patterns

Robust inversion and detection techniques for improved imaging performance

Thesis (Ph.D.)--Boston UniversityIn this thesis we aim to improve the performance of information extraction from imaging systems through three thrusts. First, we develop improved image formation methods for physics-based, complex-valued sensing problems. We propose a regularized inversion method that incorporates prior information about the underlying field into the inversion framework for ultrasound imaging. We use experimental ultrasound data to compute inversion results with the proposed formulation and compare it with conventional inversion techniques to show the robustness of the proposed technique to loss of data. Second, we propose methods that combine inversion and detection in a unified framework to improve imaging performance. This framework is applicable for cases where the underlying field is label-based such that each pixel of the underlying field can only assume values from a discrete, limited set. We consider this unified framework in the context of combinatorial optimization and propose graph-cut based methods that would result in label-based images, thereby eliminating the need for a separate detection step. Finally, we propose a robust method of object detection from microscopic nanoparticle images. In particular, we focus on a portable, low cost interferometric imaging platform and propose robust detection algorithms using tools from computer vision. We model the electromagnetic image formation process and use this model to create an enhanced detection technique. The effectiveness of the proposed technique is demonstrated using manually labeled ground-truth data. In addition, we extend these tools to develop a detection based autofocusing algorithm tailored for the high numerical aperture interferometric microscope