Structured Sparsity: Discrete and Convex approaches

Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the relevant information therein typically resides in a much lower dimensional space. However, many solutions proposed nowadays do not leverage the true underlying structure. Recent results in CS extend the simple sparsity idea to more sophisticated {\em structured} sparsity models, which describe the interdependency between the nonzero components of a signal, allowing to increase the interpretability of the results and lead to better recovery performance. In order to better understand the impact of structured sparsity, in this chapter we analyze the connections between the discrete models and their convex relaxations, highlighting their relative advantages. We start with the general group sparse model and then elaborate on two important special cases: the dispersive and the hierarchical models. For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations. We also consider more general structures as defined by set functions and present their convex proxies. Further, we discuss efficient optimization solutions for structured sparsity problems and illustrate structured sparsity in action via three applications.Comment: 30 pages, 18 figure

Large Deviations for Non-Markovian Diffusions and a Path-Dependent Eikonal Equation

This paper provides a large deviation principle for Non-Markovian, Brownian motion driven stochastic differential equations with random coefficients. Similar to Gao and Liu \cite{GL}, this extends the corresponding results collected in Freidlin and Wentzell \cite{FreidlinWentzell}. However, we use a different line of argument, adapting the PDE method of Fleming \cite{Fleming} and Evans and Ishii \cite{EvansIshii} to the path-dependent case, by using backward stochastic differential techniques. Similar to the Markovian case, we obtain a characterization of the action function as the unique bounded solution of a path-dependent version of the Eikonal equation. Finally, we provide an application to the short maturity asymptotics of the implied volatility surface in financial mathematics

MRF Stereo Matching with Statistical Estimation of Parameters

For about the last ten years, stereo matching in computer vision has been treated as a combinatorial optimization problem. Assuming that the points in stereo images form a Markov Random Field (MRF), a variety of combinatorial optimization algorithms has been developed to optimize their underlying cost functions. In many of these algorithms, the MRF parameters of the cost functions have often been manually tuned or heuristically determined for achieving good performance results. Recently, several algorithms for statistical, hence, automatic estimation of the parameters have been published. Overall, these algorithms perform well in labeling, but they lack in performance for handling discontinuity in labeling along the surface borders. In this dissertation, we develop an algorithm for optimization of the cost function with automatic estimation of the MRF parameters – the data and smoothness parameters. Both the parameters are estimated statistically and applied in the cost function with support of adaptive neighborhood defined based on color similarity. With the proposed algorithm, discontinuity handling with higher consistency than of the existing algorithms is achieved along surface borders. The data parameters are pre-estimated from one of the stereo images by applying a hypothesis, called noise equivalence hypothesis, to eliminate interdependency between the estimations of the data and smoothness parameters. The smoothness parameters are estimated applying a combination of maximum likelihood and disparity gradient constraint, to eliminate nested inference for the estimation. The parameters for handling discontinuities in data and smoothness are defined statistically as well. We model cost functions to match the images symmetrically for improved matching performance and also to detect occlusions. Finally, we fill the occlusions in the disparity map by applying several existing and proposed algorithms and show that our best proposed segmentation based least squares algorithm performs better than the existing algorithms. We conduct experiments with the proposed algorithm on publicly available ground truth test datasets provided by the Middlebury College. Experiments show that results better than the existing algorithms’ are delivered by the proposed algorithm having the MRF parameters estimated automatically. In addition, applying the parameter estimation technique in existing stereo matching algorithm, we observe significant improvement in computational time

Mathematical Imaging and Surface Processing

Within the last decade image and geometry processing have become increasingly rigorous with solid foundations in mathematics. Both areas are research fields at the intersection of different mathematical disciplines, ranging from geometry and calculus of variations to PDE analysis and numerical analysis. The workshop brought together scientists from all these areas and a fruitful interplay took place. There was a lively exchange of ideas between geometry and image processing applications areas, characterized in a number of ways in this workshop. For example, optimal transport, first applied in computer vision is now used to define a distance measure between 3d shapes, spectral analysis as a tool in image processing can be applied in surface classification and matching, and so on. We have also seen the use of Riemannian geometry as a powerful tool to improve the analysis of multivalued images. This volume collects the abstracts for all the presentations covering this wide spectrum of tools and application domains

Directed Acyclic Graph Continuous Max-Flow Image Segmentation for Unconstrained Label Orderings

Label ordering, the specification of subset–superset relationships for segmentation labels, has been of increasing interest in image segmentation as they allow for complex regions to be represented as a collection of simple parts. Recent advances in continuous max-flow segmentation have widely expanded the possible label orderings from binary background/foreground problems to extendable frameworks in which the label ordering can be specified. This article presents Directed Acyclic Graph Max-Flow image segmentation which is flexible enough to incorporate any label ordering without constraints. This framework uses augmented Lagrangian multipliers and primal–dual optimization to develop a highly parallelized solver implemented using GPGPU. This framework is validated on synthetic, natural, and medical images illustrating its general applicability

Doctor of Philosophy

dissertationImage segmentation entails the partitioning of an image domain, usually two or three dimensions, so that each partition or segment has some meaning that is relevant to the application at hand. Accurate image segmentation is a crucial challenge in many disciplines, including medicine, computer vision, and geology. In some applications, heterogeneous pixel intensities; noisy, ill-defined, or diffusive boundaries; and irregular shapes with high variability can make it challenging to meet accuracy requirements. Various segmentation approaches tackle such challenges by casting the segmentation problem as an energy-minimization problem, and solving it using efficient optimization algorithms. These approaches are broadly classified as either region-based or edge (surface)-based depending on the features on which they operate. The focus of this dissertation is on the development of a surface-based energy model, the design of efficient formulations of optimization frameworks to incorporate such energy, and the solution of the energy-minimization problem using graph cuts. This dissertation utilizes a set of four papers whose motivation is the efficient extraction of the left atrium wall from the late gadolinium enhancement magnetic resonance imaging (LGE-MRI) image volume. This dissertation utilizes these energy formulations for other applications, including contact lens segmentation in the optical coherence tomography (OCT) data and the extraction of geologic features in seismic data. Chapters 2 through 5 (papers 1 through 4) explore building a surface-based image segmentation model by progressively adding components to improve its accuracy and robustness. The first paper defines a parametric search space and its discrete formulation in the form of a multilayer three-dimensional mesh model within which the segmentation takes place. It includes a generative intensity model, and we optimize using a graph formulation of the surface net problem. The second paper proposes a Bayesian framework with a Markov random field (MRF) prior that gives rise to another class of surface nets, which provides better segmentation with smooth boundaries. The third paper presents a maximum a posteriori (MAP)-based surface estimation framework that relies on a generative image model by incorporating global shape priors, in addition to the MRF, within the Bayesian formulation. Thus, the resulting surface not only depends on the learned model of shapes,but also accommodates the test data irregularities through smooth deviations from these priors. Further, the paper proposes a new shape parameter estimation scheme, in closed form, for segmentation as a part of the optimization process. Finally, the fourth paper (under review at the time of this document) presents an extensive analysis of the MAP framework and presents improved mesh generation and generative intensity models. It also performs a thorough analysis of the segmentation results that demonstrates the effectiveness of the proposed method qualitatively, quantitatively, and clinically. Chapter 6, consisting of unpublished work, demonstrates the application of an MRF-based Bayesian framework to segment coupled surfaces of contact lenses in optical coherence tomography images. This chapter also shows an application related to the extraction of geological structures in seismic volumes. Due to the large sizes of seismic volume datasets, we also present fast, approximate surface-based energy minimization strategies that achieve better speed-ups and memory consumption

확률론적 순차적 그래프 근사를 이용한 MRF 최적화

학위논문 (석사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 이경무.Markov random elds have been powerful models in computer vision but tractable algorithms to obtain exact solution for the corresponding energy functions are lim- itedapproximate solutions, in most cases are provided for efficiency. In this work graduated optimization technique is applied in a novel way to develop an efficient al- gorithm for solving general multi-label MRF optimization problem called Stochastic Graduated graph approximation (SGGA) algorithm. The algorithm initially min- imizes a simplied function and progressively transforms that function until it is equivalent to the original function. However, it is hard to nd how to generate the sequence of intermediate functions and what parameter to use for making transition from one problem to another. For this we propose a new iterative method of build- ing the sequence of approximations for the original energy function. We exploit a stochastic method to generate intermediate functions, which guides the intermedi- ate solutions to the near-optimal solution for the original problem. The transition from one intermediate problem to another is controlled by the schedule of gradual addition of edges. In each iteration, a deterministic algorithm such as block ICM is applied to minimize intermediate functions and to generate initial solution for the next problem. The proposed algorithm guarantees the convergence of local mini- mum. We test on a synthetic image deconvolution problem and also on the set of experiments with the OpenGM2 benchmark.Abstract i Contents iii List of Figures iv List of Tables viii 1 Introduction 2 1.1 Background of research . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Outline of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Related works 8 2.1 Graduated optimization . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Sequential Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . 11 3 Stochastic graduated graph approximation 13 3.1 Graph approximation by scanlines . . . . . . . . . . . . . . . . . . . 13 3.2 Graph approximation by trees . . . . . . . . . . . . . . . . . . . . . . 18 4 Minimization of intermediate energy functions 20 4.1 Block Iterated conditional modes . . . . . . . . . . . . . . . . . . . . 20 4.1.1 Block Iterated conditional modes: general idea . . . . . . . . 20 4.1.2 Block ICM for graduated graph approximation . . . . . . . . 21 4.2 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.1 Dynamic programming: general idea . . . . . . . . . . . . . . 23 4.2.2 The DP algorithm on scanlines for graduated graph approxi- mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.2.3 The DP algorithm on trees . . . . . . . . . . . . . . . . . . . 27 5 Experiments 29 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2 Image deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.3 OpenGM2 benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6 Conclusion 51 Bibliography 52 59 60Maste

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)

The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist1