Geometric deep learning: going beyond Euclidean data

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them. Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field

Sharp isoperimetric inequalities via the ABP

Given an arbitrary convex cone of Rn, we find a geometric class of homogeneous weights for which balls centered at the origin and intersected with the cone are minimizers of the weighted isoperimetric problem in the convex cone. This leads to isoperimetric inequalities with the optimal constant that were unknown even for a sector of the plane. Our result applies to all nonnegative homogeneous weights in Rn satisfying a concavity condition in the cone. The condition is equivalent to a natural curvature-dimension bound and also to the nonnegativeness of a Bakry-Emery Ricci tensor. Even that our weights are nonradial, still balls are minimizers of the weighted isoperimetric problem. A particular important case is that of monomial weights. Our proof uses the ABP method applied to an appropriate linear Neumann problem. We also study the anisotropic isoperimetric problem in convex cones for the same class of weights. We prove that the Wulff shape (intersected with the cone) minimizes the anisotropic weighted perimeter under the weighted volume constraint. As a particular case of our results, we give new proofs of two classical results: the Wulff inequality and the isoperimetric inequality in convex cones of Lions and PacellaPeer ReviewedPostprint (published version

Detail-preserving and Content-aware Variational Multi-view Stereo Reconstruction

Accurate recovery of 3D geometrical surfaces from calibrated 2D multi-view images is a fundamental yet active research area in computer vision. Despite the steady progress in multi-view stereo reconstruction, most existing methods are still limited in recovering fine-scale details and sharp features while suppressing noises, and may fail in reconstructing regions with few textures. To address these limitations, this paper presents a Detail-preserving and Content-aware Variational (DCV) multi-view stereo method, which reconstructs the 3D surface by alternating between reprojection error minimization and mesh denoising. In reprojection error minimization, we propose a novel inter-image similarity measure, which is effective to preserve fine-scale details of the reconstructed surface and builds a connection between guided image filtering and image registration. In mesh denoising, we propose a content-aware $\ell_{p}$-minimization algorithm by adaptively estimating the $p$ value and regularization parameters based on the current input. It is much more promising in suppressing noise while preserving sharp features than conventional isotropic mesh smoothing. Experimental results on benchmark datasets demonstrate that our DCV method is capable of recovering more surface details, and obtains cleaner and more accurate reconstructions than state-of-the-art methods. In particular, our method achieves the best results among all published methods on the Middlebury dino ring and dino sparse ring datasets in terms of both completeness and accuracy.Comment: 14 pages,16 figures. Submitted to IEEE Transaction on image processin

Gaussian Process Morphable Models

Statistical shape models (SSMs) represent a class of shapes as a normal distribution of point variations, whose parameters are estimated from example shapes. Principal component analysis (PCA) is applied to obtain a low-dimensional representation of the shape variation in terms of the leading principal components. In this paper, we propose a generalization of SSMs, called Gaussian Process Morphable Models (GPMMs). We model the shape variations with a Gaussian process, which we represent using the leading components of its Karhunen-Loeve expansion. To compute the expansion, we make use of an approximation scheme based on the Nystrom method. The resulting model can be seen as a continuous analogon of an SSM. However, while for SSMs the shape variation is restricted to the span of the example data, with GPMMs we can define the shape variation using any Gaussian process. For example, we can build shape models that correspond to classical spline models, and thus do not require any example data. Furthermore, Gaussian processes make it possible to combine different models. For example, an SSM can be extended with a spline model, to obtain a model that incorporates learned shape characteristics, but is flexible enough to explain shapes that cannot be represented by the SSM. We introduce a simple algorithm for fitting a GPMM to a surface or image. This results in a non-rigid registration approach, whose regularization properties are defined by a GPMM. We show how we can obtain different registration schemes,including methods for multi-scale, spatially-varying or hybrid registration, by constructing an appropriate GPMM. As our approach strictly separates modelling from the fitting process, this is all achieved without changes to the fitting algorithm. We show the applicability and versatility of GPMMs on a clinical use case, where the goal is the model-based segmentation of 3D forearm images

Universal spatial correlations in the anisotropic Kondo screening cloud: analytical insights and numerically exact results from a coherent state expansion

We analyze the spatial correlations in the spin density of an electron gas in the vicinity of a Kondo impurity. Our analysis extends to the spin-anisotropic regime, which was not investigated in the literature. We use an original and numerically exact method, based on a systematic coherent-state expansion of the ground state of the underlying spin-boson Hamiltonian, which we apply to the computation of observables that are specific to the fermionic Kondo model. We also present an important technical improvement to the method, that obviates the need to discretize modes of the Fermi sea, and allows one to tackle the problem in the thermodynamic limit. One can thus obtain excellent spatial resolution over arbitrary length scales, for a relatively low computational cost, a feature that gives the method an advantage over popular techniques such as NRG and DMRG. We find that the anisotropic Kondo model shows rich universal scaling behavior in the spatial structure of the entanglement cloud. First, SU(2) spin-symmetry is dynamically restored in a finite domain in parameter space in vicinity of the isotropic line, as expected from poor man's scaling. We are also able to obtain in closed analytical form a set of different, yet universal, scaling curves for strong exchange asymmetry, which are parametrized by the longitudinal exchange coupling. Deep inside the cloud, i.e. for distances smaller than the Kondo length, the correlation between the electron spin density and the impurity spin oscillates between ferromagnetic and antiferromagnetic values at the scale of the Fermi wavelength, an effect that is drastically enhanced at strongly anisotropic couplings. Our results also provide further numerical checks and alternative analytical approximations for the recently computed Kondo overlaps [PRL 114, 080601 (2015)].Comment: 27 pages + 2 pages of Supplementary materials. The manuscript was largely extended in V2, and contains now a comparison to the Toulouse limit, and well as a detailed study of the restoration of SU(2) symmetry. The displayed html abstract has been shortened compared to the pdf versio

Geometric deep learning for shape analysis: extending deep learning techniques to non-Euclidean manifolds

The past decade in computer vision research has witnessed the re-emergence of artificial neural networks (ANN), and in particular convolutional neural network (CNN) techniques, allowing to learn powerful feature representations from large collections of data. Nowadays these techniques are better known under the umbrella term deep learning and have achieved a breakthrough in performance in a wide range of image analysis applications such as image classification, segmentation, and annotation. Nevertheless, when attempting to apply deep learning paradigms to 3D shapes one has to face fundamental differences between images and geometric objects. The main difference between images and 3D shapes is the non-Euclidean nature of the latter. This implies that basic operations, such as linear combination or convolution, that are taken for granted in the Euclidean case, are not even well defined on non-Euclidean domains. This happens to be the major obstacle that so far has precluded the successful application of deep learning methods on non-Euclidean geometric data. The goal of this thesis is to overcome this obstacle by extending deep learning tecniques (including, but not limiting to CNNs) to non-Euclidean domains. We present different approaches providing such extension and test their effectiveness in the context of shape similarity and correspondence applications. The proposed approaches are evaluated on several challenging experiments, achieving state-of-the- art results significantly outperforming other methods. To the best of our knowledge, this thesis presents different original contributions. First, this work pioneers the generalization of CNNs to discrete manifolds. Second, it provides an alternative formulation of the spectral convolution operation in terms of the windowed Fourier transform to overcome the drawbacks of the Fourier one. Third, it introduces a spatial domain formulation of convolution operation using patch operators and several ways of their construction (geodesic, anisotropic diffusion, mixture of Gaussians). Fourth, at the moment of publication the proposed approaches achieved state-of-the-art results in different computer graphics and vision applications such as shape descriptors and correspondence

Piecewise smooth reconstruction of normal vector field on digital data

International audienceWe propose a novel method to regularize a normal vector field defined on a digital surface (boundary of a set of voxels). When the digital surface is a digitization of a piecewise smooth manifold, our method localizes sharp features (edges) while regularizing the input normal vector field at the same time. It relies on the optimisation of a variant of the Ambrosio-Tortorelli functional, originally defined for denoising and contour extraction in image processing [AT90]. We reformulate this functional to digital surface processing thanks to discrete calculus operators. Experiments show that the output normal field is very robust to digitization artifacts or noise, and also fairly independent of the sampling resolution. The method allows the user to choose independently the amount of smoothing and the length of the set of discontinuities. Sharp and vanishing features are correctly delineated even on extremely damaged data. Finally, our method can be used to enhance considerably the output of state-of- the-art normal field estimators like Voronoi Covariance Measure [MOG11] or Randomized Hough Transform [BM12]

Geodesic flows on semidirect-product Lie groups: geometry of singular measure-valued solutions

The EPDiff equation (or dispersionless Camassa-Holm equation in 1D) is a well known example of geodesic motion on the Diff group of smooth invertible maps (diffeomorphisms). Its recent two-component extension governs geodesic motion on the semidirect product ${\rm Diff}\circledS{\cal F}$, where $\mathcal{F}$ denotes the space of scalar functions. This paper generalizes the second construction to consider geodesic motion on ${\rm Diff} \circledS\mathfrak{g}$, where $\mathfrak{g}$ denotes the space of scalar functions that take values on a certain Lie algebra (for example, $\mathfrak{g}=\mathcal{F}\otimes\mathfrak{so}(3)$). Measure-valued delta-like solutions are shown to be momentum maps possessing a dual pair structure, thereby extending previous results for the EPDiff equation. The collective Hamiltonians are shown to fit into the Kaluza-Klein theory of particles in a Yang-Mills field and these formulations are shown to apply also at the continuum PDE level. In the continuum description, the Kaluza-Klein approach produces the Kelvin circulation theorem.Comment: 22 pages, 2 figures. Submitted to Proc. R. Soc.