Graph Spectral Image Processing

Recent advent of graph signal processing (GSP) has spurred intensive studies of signals that live naturally on irregular data kernels described by graphs (e.g., social networks, wireless sensor networks). Though a digital image contains pixels that reside on a regularly sampled 2D grid, if one can design an appropriate underlying graph connecting pixels with weights that reflect the image structure, then one can interpret the image (or image patch) as a signal on a graph, and apply GSP tools for processing and analysis of the signal in graph spectral domain. In this article, we overview recent graph spectral techniques in GSP specifically for image / video processing. The topics covered include image compression, image restoration, image filtering and image segmentation

Semantic 3D Reconstruction with Finite Element Bases

We propose a novel framework for the discretisation of multi-label problems on arbitrary, continuous domains. Our work bridges the gap between general FEM discretisations, and labeling problems that arise in a variety of computer vision tasks, including for instance those derived from the generalised Potts model. Starting from the popular formulation of labeling as a convex relaxation by functional lifting, we show that FEM discretisation is valid for the most general case, where the regulariser is anisotropic and non-metric. While our findings are generic and applicable to different vision problems, we demonstrate their practical implementation in the context of semantic 3D reconstruction, where such regularisers have proved particularly beneficial. The proposed FEM approach leads to a smaller memory footprint as well as faster computation, and it constitutes a very simple way to enable variable, adaptive resolution within the same model

3D Shape Understanding and Generation

In recent years, Machine Learning techniques have revolutionized solutions to longstanding image-based problems, like image classification, generation, semantic segmentation, object detection and many others. However, if we want to be able to build agents that can successfully interact with the real world, those techniques need to be capable of reasoning about the world as it truly is: a tridimensional space. There are two main challenges while handling 3D information in machine learning models. First, it is not clear what is the best 3D representation. For images, convolutional neural networks (CNNs) operating on raster images yield the best results in virtually all image-based benchmarks. For 3D data, the best combination of model and representation is still an open question. Second, 3D data is not available on the same scale as images – taking pictures is a common procedure in our daily lives, whereas capturing 3D content is an activity usually restricted to specialized professionals. This thesis is focused on addressing both of these issues. Which model and representation should we use for generating and recognizing 3D data? What are efficient ways of learning 3D representations from a few examples? Is it possible to leverage image data to build models capable of reasoning about the world in 3D? Our research findings show that it is possible to build models that efficiently generate 3D shapes as irregularly structured representations. Those models require significantly less memory while generating higher quality shapes than the ones based on voxels and multi-view representations. We start by developing techniques to generate shapes represented as point clouds. This class of models leads to high quality reconstructions and better unsupervised feature learning. However, since point clouds are not amenable to editing and human manipulation, we also present models capable of generating shapes as sets of shape handles -- simpler primitives that summarize complex 3D shapes and were specifically designed for high-level tasks and user interaction. Despite their effectiveness, those approaches require some form of 3D supervision, which is scarce. We present multiple alternatives to this problem. First, we investigate how approximate convex decomposition techniques can be used as self-supervision to improve recognition models when only a limited number of labels are available. Second, we study how neural network architectures induce shape priors that can be used in multiple reconstruction tasks -- using both volumetric and manifold representations. In this regime, reconstruction is performed from a single example -- either a sparse point cloud or multiple silhouettes. Finally, we demonstrate how to train generative models of 3D shapes without using any 3D supervision by combining differentiable rendering techniques and Generative Adversarial Networks

Markov random field image modelling

Includes bibliographical references.This work investigated some of the consequences of using a priori information in image processing using computer tomography (CT) as an example. Prior information is information about the solution that is known apart from measurement data. This information can be represented as a probability distribution. In order to define a probability density distribution in high dimensional problems like those found in image processing it becomes necessary to adopt some form of parametric model for the distribution. Markov random fields (MRFs) provide just such a vehicle for modelling the a priori distribution of labels found in images. In particular, this work investigated the suitability of MRF models for modelling a priori information about the distribution of attenuation coefficients found in CT scans

VTrails: Inferring Vessels with Geodesic Connectivity Trees

The analysis of vessel morphology and connectivity has an impact on a number of cardiovascular and neurovascular applications by providing patient-specific high-level quantitative features such as spatial location, direction and scale. In this paper we present an end-to-end approach to extract an acyclic vascular tree from angiographic data by solving a connectivity-enforcing anisotropic fast marching over a voxel-wise tensor field representing the orientation of the underlying vascular tree. The method is validated using synthetic and real vascular images. We compare VTrails against classical and state-of-the-art ridge detectors for tubular structures by assessing the connectedness of the vesselness map and inspecting the synthesized tensor field as proof of concept. VTrails performance is evaluated on images with different levels of degradation: we verify that the extracted vascular network is an acyclic graph (i.e. a tree), and we report the extraction accuracy, precision and recall

Quantification of cortical folding using MR image data

The cerebral cortex is a thin layer of tissue lining the brain where neural circuits perform important high level functions including sensory perception, motor control and language processing. In the third trimester the fetal cortex folds rapidly from a smooth sheet into a highly convoluted arrangement of gyri and sulci. Premature birth is a high risk factor for poor neurodevelopmental outcome and has been associated with abnormal cortical development, however the nature of the disruption to developmental processes is not fully understood. Recent developments in magnetic resonance imaging have allowed the acquisition of high quality brain images of preterms and also fetuses in-utero. The aim of this thesis is to develop techniques which quantify folding from these images in order to better understand cortical development in these two populations. A framework is presented that quantifies global and regional folding using curvature-based measures. This methodology was applied to fetuses over a wide gestational age range (21.7 to 38.9 weeks) for a large number of subjects (N = 80) extending our understanding of how the cortex folds through this critical developmental period. The changing relationship between the folding measures and gestational age was modelled with a Gompertz function which allowed an accurate prediction of physiological age. A spectral-based method is outlined for constructing a spatio-temporal surface atlas (a sequence of mean cortical surface meshes for weekly intervals). A key advantage of this method is the ability to do group-wise atlasing without bias to the anatomy of an initial reference subject. Mean surface templates were constructed for both fetuses and preterms allowing a preliminary comparison of mean cortical shape over the postmenstrual age range 28-36 weeks. Displacement patterns were revealed which intensified with increasing prematurity, however more work is needed to evaluate the reliability of these findings.Open Acces

Approches tomographiques structurelles pour l'analyse du milieu urbain par tomographie SAR THR : TomoSAR

SAR tomography consists in exploiting multiple images from the same area acquired from a slightly different angle to retrieve the 3-D distribution of the complex reflectivity on the ground. As the transmitted waves are coherent, the desired spatial information (along with the vertical axis) is coded in the phase of the pixels. Many methods have been proposed to retrieve this information in the past years. However, the natural redundancies of the scene are generally not exploited to improve the tomographic estimation step. This Ph.D. presents new approaches to regularize the estimated reflectivity density obtained through SAR tomography by exploiting the urban geometrical structures.La tomographie SAR exploite plusieurs acquisitions d'une même zone acquises d'un point de vue légerement différent pour reconstruire la densité complexe de réflectivité au sol. Cette technique d'imagerie s'appuyant sur l'émission et la réception d'ondes électromagnétiques cohérentes, les données analysées sont complexes et l'information spatiale manquante (selon la verticale) est codée dans la phase. De nombreuse méthodes ont pu être proposées pour retrouver cette information. L'utilisation des redondances naturelles à certains milieux n'est toutefois généralement pas exploitée pour améliorer l'estimation tomographique. Cette thèse propose d'utiliser l'information structurelle propre aux structures urbaines pour régulariser les densités de réflecteurs obtenues par cette technique