Graph Spectral Image Processing

Recent advent of graph signal processing (GSP) has spurred intensive studies of signals that live naturally on irregular data kernels described by graphs (e.g., social networks, wireless sensor networks). Though a digital image contains pixels that reside on a regularly sampled 2D grid, if one can design an appropriate underlying graph connecting pixels with weights that reflect the image structure, then one can interpret the image (or image patch) as a signal on a graph, and apply GSP tools for processing and analysis of the signal in graph spectral domain. In this article, we overview recent graph spectral techniques in GSP specifically for image / video processing. The topics covered include image compression, image restoration, image filtering and image segmentation

Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs

In this work we propose a structured prediction technique that combines the virtues of Gaussian Conditional Random Fields (G-CRF) with Deep Learning: (a) our structured prediction task has a unique global optimum that is obtained exactly from the solution of a linear system (b) the gradients of our model parameters are analytically computed using closed form expressions, in contrast to the memory-demanding contemporary deep structured prediction approaches that rely on back-propagation-through-time, (c) our pairwise terms do not have to be simple hand-crafted expressions, as in the line of works building on the DenseCRF, but can rather be `discovered' from data through deep architectures, and (d) out system can trained in an end-to-end manner. Building on standard tools from numerical analysis we develop very efficient algorithms for inference and learning, as well as a customized technique adapted to the semantic segmentation task. This efficiency allows us to explore more sophisticated architectures for structured prediction in deep learning: we introduce multi-resolution architectures to couple information across scales in a joint optimization framework, yielding systematic improvements. We demonstrate the utility of our approach on the challenging VOC PASCAL 2012 image segmentation benchmark, showing substantial improvements over strong baselines. We make all of our code and experiments available at {https://github.com/siddharthachandra/gcrf}Comment: Our code is available at https://github.com/siddharthachandra/gcr

확률론적 순차적 그래프 근사를 이용한 MRF 최적화

학위논문 (석사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 이경무.Markov random elds have been powerful models in computer vision but tractable algorithms to obtain exact solution for the corresponding energy functions are lim- itedapproximate solutions, in most cases are provided for efficiency. In this work graduated optimization technique is applied in a novel way to develop an efficient al- gorithm for solving general multi-label MRF optimization problem called Stochastic Graduated graph approximation (SGGA) algorithm. The algorithm initially min- imizes a simplied function and progressively transforms that function until it is equivalent to the original function. However, it is hard to nd how to generate the sequence of intermediate functions and what parameter to use for making transition from one problem to another. For this we propose a new iterative method of build- ing the sequence of approximations for the original energy function. We exploit a stochastic method to generate intermediate functions, which guides the intermedi- ate solutions to the near-optimal solution for the original problem. The transition from one intermediate problem to another is controlled by the schedule of gradual addition of edges. In each iteration, a deterministic algorithm such as block ICM is applied to minimize intermediate functions and to generate initial solution for the next problem. The proposed algorithm guarantees the convergence of local mini- mum. We test on a synthetic image deconvolution problem and also on the set of experiments with the OpenGM2 benchmark.Abstract i Contents iii List of Figures iv List of Tables viii 1 Introduction 2 1.1 Background of research . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Outline of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Related works 8 2.1 Graduated optimization . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Sequential Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . 11 3 Stochastic graduated graph approximation 13 3.1 Graph approximation by scanlines . . . . . . . . . . . . . . . . . . . 13 3.2 Graph approximation by trees . . . . . . . . . . . . . . . . . . . . . . 18 4 Minimization of intermediate energy functions 20 4.1 Block Iterated conditional modes . . . . . . . . . . . . . . . . . . . . 20 4.1.1 Block Iterated conditional modes: general idea . . . . . . . . 20 4.1.2 Block ICM for graduated graph approximation . . . . . . . . 21 4.2 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.2.1 Dynamic programming: general idea . . . . . . . . . . . . . . 23 4.2.2 The DP algorithm on scanlines for graduated graph approxi- mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.2.3 The DP algorithm on trees . . . . . . . . . . . . . . . . . . . 27 5 Experiments 29 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2 Image deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.3 OpenGM2 benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6 Conclusion 51 Bibliography 52 59 60Maste

Graph Signal Processing: Overview, Challenges and Applications

Research in Graph Signal Processing (GSP) aims to develop tools for processing data defined on irregular graph domains. In this paper we first provide an overview of core ideas in GSP and their connection to conventional digital signal processing. We then summarize recent developments in developing basic GSP tools, including methods for sampling, filtering or graph learning. Next, we review progress in several application areas using GSP, including processing and analysis of sensor network data, biological data, and applications to image processing and machine learning. We finish by providing a brief historical perspective to highlight how concepts recently developed in GSP build on top of prior research in other areas.Comment: To appear, Proceedings of the IEE

Statistical models for natural scene data

This thesis considers statistical modelling of natural image data. Obtaining advances in this field can have significant impact for both engineering applications, and for the understanding of the human visual system. Several recent advances in natural image modelling have been obtained with the use of unsupervised feature learning. We consider a class of such models, restricted Boltzmann machines (RBMs), used in many recent state-of-the-art image models. We develop extensions of these stochastic artificial neural networks, and use them as a basis for building more effective image models, and tools for computational vision. We first develop a novel framework for obtaining Boltzmann machines, in which the hidden unit activations co-transform with transformed input stimuli in a stable and predictable way throughout the network. We define such models to be transformation equivariant. Such properties have been shown useful for computer vision systems, and have been motivational for example in the development of steerable filters, a widely used classical feature extraction technique. Translation equivariant feature sharing has been the standard method for scaling image models beyond patch-sized data to large images. In our framework we extend shallow and deep models to account for other kinds of transformations as well, focusing on in-plane rotations. Motivated by the unsatisfactory results of current generative natural image models, we take a step back, and evaluate whether they are able to model a subclass of the data, natural image textures. This is a necessary subcomponent of any credible model for visual scenes. We assess the performance of a state- of-the-art model of natural images for texture generation, using a dataset and evaluation techniques from in prior work. We also perform a dissection of the model architecture, uncovering the properties important for good performance. Building on this, we develop structured extensions for more complicated data comprised of textures from multiple classes, using the single-texture model architecture as a basis. These models are shown to be able to produce state-of-the-art texture synthesis results quantitatively, and are also effective qualitatively. It is demonstrated empirically that the developed multiple-texture framework provides a means to generate images of differently textured regions, more generic globally varying textures, and can also be used for texture interpolation, where the approach is radically dfferent from the others in the area. Finally we consider visual boundary prediction from natural images. The work aims to improve understanding of Boltzmann machines in the generation of image segment boundaries, and to investigate deep neural network architectures for learning the boundary detection problem. The developed networks (which avoid several hand-crafted model and feature designs commonly used for the problem), produce the fastest reported inference times in the literature, combined with state-of-the-art performance

Bayesian image reconstruction and adaptive scene sampling in single-photon LiDAR imaging

Three-Dimensional multispectral Light Detection And Ranging (LiDAR) used with time-correlated Single-Photon (SP) detection has emerged as a key imaging modality for high-resolution depth imaging due to its high sensitivity and excellent surface-to-surface resolution. This allowed depth imaging through adversarial conditions with a prime role in numerous applications. However, several practical challenges currently limit the use of LiDAR in real-world conditions. Large data volume constitutes a major challenge for multispectral SP-LiDAR imaging due to the acquisition of millions of events per second that are usually gathered in large histogram cubes. This challenge is more evident when the useful signal photons are attenuated and the background noise is amplified as a result of imaging through a scattering environment such as underwater or fog. Another limitation includes the detection of multiple-surfaces-per pixel which usually occurs when imaging through semi-transparent materials (e.g., windows, camouflage), or in long-range profiling. This thesis proposes robust and fast computational solutions to improve the acquisition and processing of LiDAR data while measuring uncertainty on high-dimensional data. A smart task-based sampling framework is proposed to improve the acquisition process and reduce data volume. In addition, the processing was improved using a Bayesian approach to different types of inverse problems (e.g. spectral classification, and scene reconstruction). The contributions of this thesis enables fast and robust 3D reconstruction of complex scenes, paving the way for the extensive use of single-photon imaging in real-world applications