Time Discrete Geodesic Paths in the Space of Images

In this paper the space of images is considered as a Riemannian manifold using the metamorphosis approach, where the underlying Riemannian metric simultaneously measures the cost of image transport and intensity variation. A robust and effective variational time discretization of geodesics paths is proposed. This requires to minimize a discrete path energy consisting of a sum of consecutive image matching functionals over a set of image intensity maps and pairwise matching deformations. For square-integrable input images the existence of discrete, connecting geodesic paths defined as minimizers of this variational problem is shown. Furthermore, $\Gamma$-convergence of the underlying discrete path energy to the continuous path energy is proved. This includes a diffeomorphism property for the induced transport and the existence of a square-integrable weak material derivative in space and time. A spatial discretization via finite elements combined with an alternating descent scheme in the set of image intensity maps and the set of matching deformations is presented to approximate discrete geodesic paths numerically. Computational results underline the efficiency of the proposed approach and demonstrate important qualitative properties.Comment: 27 pages, 7 figure

Metric combinatorics of convex polyhedra: cut loci and nonoverlapping unfoldings

This paper is a study of the interaction between the combinatorics of boundaries of convex polytopes in arbitrary dimension and their metric geometry. Let S be the boundary of a convex polytope of dimension d+1, or more generally let S be a `convex polyhedral pseudomanifold'. We prove that S has a polyhedral nonoverlapping unfolding into R^d, so the metric space S is obtained from a closed (usually nonconvex) polyhedral ball in R^d by identifying pairs of boundary faces isometrically. Our existence proof exploits geodesic flow away from a source point v in S, which is the exponential map to S from the tangent space at v. We characterize the `cut locus' (the closure of the set of points in S with more than one shortest path to v) as a polyhedral complex in terms of Voronoi diagrams on facets. Analyzing infinitesimal expansion of the wavefront consisting of points at constant distance from v on S produces an algorithmic method for constructing Voronoi diagrams in each facet, and hence the unfolding of S. The algorithm, for which we provide pseudocode, solves the discrete geodesic problem. Its main construction generalizes the source unfolding for boundaries of 3-polytopes into R^2. We present conjectures concerning the number of shortest paths on the boundaries of convex polyhedra, and concerning continuous unfolding of convex polyhedra. We also comment on the intrinsic non-polynomial complexity of nonconvex polyhedral manifolds.Comment: 47 pages; 21 PostScript (.eps) figures, most in colo

The Riemannian Geometry of Deep Generative Models

Deep generative models learn a mapping from a low dimensional latent space to a high-dimensional data space. Under certain regularity conditions, these models parameterize nonlinear manifolds in the data space. In this paper, we investigate the Riemannian geometry of these generated manifolds. First, we develop efficient algorithms for computing geodesic curves, which provide an intrinsic notion of distance between points on the manifold. Second, we develop an algorithm for parallel translation of a tangent vector along a path on the manifold. We show how parallel translation can be used to generate analogies, i.e., to transport a change in one data point into a semantically similar change of another data point. Our experiments on real image data show that the manifolds learned by deep generative models, while nonlinear, are surprisingly close to zero curvature. The practical implication is that linear paths in the latent space closely approximate geodesics on the generated manifold. However, further investigation into this phenomenon is warranted, to identify if there are other architectures or datasets where curvature plays a more prominent role. We believe that exploring the Riemannian geometry of deep generative models, using the tools developed in this paper, will be an important step in understanding the high-dimensional, nonlinear spaces these models learn.Comment: 9 page

Discrete Riemannian Calculus and A Posteriori Error Control on Shape Spaces

In this thesis, a novel discrete approximation of the curvature tensor on Riemannian manifolds is derived, efficient methods to interpolate and extrapolate images in the context of the time discrete metamorphosis model are analyzed, and an a posteriori error estimator for the binary Mumford–Shah model is examined. Departing from the variational time discretization on (possibly infinite-dimensional) Riemannian manifolds originally proposed by Rumpf and Wirth, in which a consistent time discrete approximation of geodesic curves, the logarithm, the exponential map and parallel transport is analyzed, we construct the discrete curvature tensor and prove its convergence under certain smoothness assumptions. To this end, several time discrete parallel transports are applied to suitably rescaled tangent vectors, where each parallel transport is computed using Schild’s ladder. The associated convergence proof essentially relies on multiple Taylor expansions incorporating symmetry and scaling relations. In several numerical examples we validate this approach for surfaces. The by now classical flow of diffeomorphism approach allows the transport of image intensities along paths in time, which are characterized by diffeomorphisms, and the brightness of each image particle is assumed to be constant along each trajectory. As an extension, the metamorphosis model proposed by Trouvé, Younes and coworkers allows for intensity variations of the image particles along the paths, which is reflected by an additional penalization term appearing in the energy functional that quantifies the squared weak material derivative. Taking into account the aforementioned time discretization, we propose a time discrete metamorphosis model in which the associated time discrete path energy consists of the sum of squared L2-mismatch functionals of successive square-integrable image intensity functions and a regularization functional for pairwise deformations. Our main contributions are the existence proof of time discrete geodesic curves in the context of this model, which are defined as minimizers of the time discrete path energy, and the proof of the Mosco-convergence of a suitable interpolation of the time discrete to the time continuous path energy with respect to the L2-topology. Using an alternating update scheme as well as a multilinear finite element respectively cubic spline discretization for the images and deformations allows to efficiently compute time discrete geodesic curves. In several numerical examples we demonstrate that time discrete geodesics can be robustly computed for gray-scale and color images. Taking into account the time discretization of the metamorphosis model we define the discrete exponential map in the space of images, which allows image extrapolation of arbitrary length for given weakly differentiable initial images and variations. To this end, starting from a suitable reformulation of the Euler–Lagrange equations characterizing the one-step extrapolation a fixed point iteration is employed to establish the existence of critical points of the Euler–Lagrange equations provided that the initial variation is small in L2. In combination with an implicit function type argument requiring H1-closeness of the initial variation one can prove the local existence as well as the local uniqueness of the discrete exponential map. The numerical algorithm for the one-step extrapolation is based on a slightly modified fixed point iteration using a spatial Galerkin scheme to obtain the optimal deformation associated with the unknown image, from which the unknown image itself can be recovered. To prove the applicability of the proposed method we compute the extrapolated image path for real image data. A common tool to segment images and shapes into multiple regions was developed by Mumford and Shah. The starting point to derive a posteriori error estimates for the binary Mumford–Shah model, which is obtained by restricting the original model to two regions, is a uniformly convex and non-constrained relaxation of the binary model following the work by Chambolle and Berkels. In particular, minimizers of the binary model can be exactly recovered from minimizers of the relaxed model via thresholding. Then, applying duality techniques proposed by Repin and Bartels allows deriving a consistent functional a posteriori error estimate for the relaxed model. Afterwards, an a posteriori error estimate for the original binary model can be computed incorporating a suitable cut-out argument in combination with the functional error estimate. To calculate minimizers of the relaxed model on an adaptive mesh described by a quadtree structure, we employ a primal-dual as well as a purely dual algorithm. The quality of the error estimator is analyzed for different gray-scale input images