Sharp Trudinger-Moser Inequalities On Riemannian Manifolds And Heisenberg Groups

Archival abstract submitte

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces

Gaussian processes are arguably the most important class of spatiotemporal models within machine learning. They encode prior information about the modeled function and can be used for exact or approximate Bayesian learning. In many applications, particularly in physical sciences and engineering, but also in areas such as geostatistics and neuroscience, invariance to symmetries is one of the most fundamental forms of prior information one can consider. The invariance of a Gaussian process' covariance to such symmetries gives rise to the most natural generalization of the concept of stationarity to such spaces. In this work, we develop constructive and practical techniques for building stationary Gaussian processes on a very large class of non-Euclidean spaces arising in the context of symmetries. Our techniques make it possible to (i) calculate covariance kernels and (ii) sample from prior and posterior Gaussian processes defined on such spaces, both in a practical manner. This work is split into two parts, each involving different technical considerations: part I studies compact spaces, while part II studies non-compact spaces possessing certain structure. Our contributions make the non-Euclidean Gaussian process models we study compatible with well-understood computational techniques available in standard Gaussian process software packages, thereby making them accessible to practitioners

Robust Bayesian Inference on Riemannian Submanifold

Non-Euclidean spaces routinely arise in modern statistical applications such as in medical imaging, robotics, and computer vision, to name a few. While traditional Bayesian approaches are applicable to such settings by considering an ambient Euclidean space as the parameter space, we demonstrate the benefits of integrating manifold structure into the Bayesian framework, both theoretically and computationally. Moreover, existing Bayesian approaches which are designed specifically for manifold-valued parameters are primarily model-based, which are typically subject to inaccurate uncertainty quantification under model misspecification. In this article, we propose a robust model-free Bayesian inference for parameters defined on a Riemannian submanifold, which is shown to provide valid uncertainty quantification from a frequentist perspective. Computationally, we propose a Markov chain Monte Carlo to sample from the posterior on the Riemannian submanifold, where the mixing time, in the large sample regime, is shown to depend only on the intrinsic dimension of the parameter space instead of the potentially much larger ambient dimension. Our numerical results demonstrate the effectiveness of our approach on a variety of problems, such as reduced-rank multiple quantile regression, principal component analysis, and Fr\'{e}chet mean estimation

Learning Neural Graph Representations in Non-Euclidean Geometries

The success of Deep Learning methods is heavily dependent on the choice of the data representation. For that reason, much of the actual effort goes into Representation Learning, which seeks to design preprocessing pipelines and data transformations that can support effective learning algorithms. The aim of Representation Learning is to facilitate the task of extracting useful information for classifiers and other predictor models. In this regard, graphs arise as a convenient data structure that serves as an intermediary representation in a wide range of problems. The predominant approach to work with graphs has been to embed them in an Euclidean space, due to the power and simplicity of this geometry. Nevertheless, data in many domains exhibit non-Euclidean features, making embeddings into Riemannian manifolds with a richer structure necessary. The choice of a metric space where to embed the data imposes a geometric inductive bias, with a direct impact on the performance of the models. This thesis is about learning neural graph representations in non-Euclidean geometries and showcasing their applicability in different downstream tasks. We introduce a toolkit formed by different graph metrics with the goal of characterizing the topology of the data. In that way, we can choose a suitable target embedding space aligned to the shape of the dataset. By virtue of the geometric inductive bias provided by the structure of the non-Euclidean manifolds, neural models can achieve higher performances with a reduced parameter footprint. As a first step, we study graphs with hierarchical structures. We develop different techniques to derive hierarchical graphs from large label inventories. Noticing the capacity of hyperbolic spaces to represent tree-like arrangements, we incorporate this information into an NLP model through hyperbolic graph embeddings and showcase the higher performance that they enable. Second, we tackle the question of how to learn hierarchical representations suited for different downstream tasks. We introduce a model that jointly learns task-specific graph embeddings from a label inventory and performs classification in hyperbolic space. The model achieves state-of-the-art results on very fine-grained labels, with a remarkable reduction of the parameter size. Next, we move to matrix manifolds to work on graphs with diverse structures and properties. We propose a general framework to implement the mathematical tools required to learn graph embeddings on symmetric spaces. These spaces are of particular interest given that they have a compound geometry that simultaneously contains Euclidean as well as hyperbolic subspaces, allowing them to automatically adapt to dissimilar features in the graph. We demonstrate a concrete implementation of the framework on Siegel spaces, showcasing their versatility on different tasks. Finally, we focus on multi-relational graphs. We devise the means to translate Euclidean and hyperbolic multi-relational graph embedding models into the space of symmetric positive definite (SPD) matrices. To do so we develop gyrocalculus in this geometry and integrate it with the aforementioned framework

Aspects of Invariant Manifold Theory and Applications

Recent years have seen a surge of interest in "data-driven" approaches to determine the equations governing complex systems. Yet in spite of modern computing advances, the high dimensionality of many systems --- such as those occurring in biology and robotics --- renders direct machine learning approaches infeasible. This dissertation develops tools for the experimental study of complex systems, based on mathematical concepts from dynamical systems theory. Our approach uses the fact that parsimonious assumptions often lead to strong insights from dynamical systems theory; such insights can be leveraged in learning algorithms to mitigate the “curse of dimensionality” and make these algorithms practical. Our first contribution concerns nonlinear oscillators. Oscillators are ubiquitous in nature, and usually associated with the existence of an "asymptotic phase" which governs the long-term dynamics of the oscillator. We show that asymptotic phase can be expressed as a line integral with respect to a uniquely defined closed differential 1-form, and provide an algorithm for estimating this "ToF" from observational data. Unlike all previously available data-driven phase estimation methods, our algorithm can: (i) use observations that are much shorter than a cycle; (ii) recover phase within the entire region for which data convergent to the limit cycle is available; (iii) recover the phase response curves (PRC-s) that govern weak oscillator coupling; (iv) show isochron curvature, and recover nonlinear features of isochron geometry. Our method may find application wherever models of oscillator dynamics need to be constructed from measured or simulated time-series. Our next contribution concerns locomotion systems which are dominated by viscous friction in the sense that without power expenditure they quickly come to a standstill. From geometric mechanics, it is known that in the ``Stokesian'' (viscous; zero Reynolds number) limit, the motion is governed by a reduced order "connection'' model that describes how body shape change produces motion for the body frame with respect to the world. In the "perturbed Stokes regime'' where inertial forces are still dominated by viscosity, but are not negligible (low Reynolds number), we show that motion is still governed by a functional relationship between shape velocity and body velocity, but this function is no longer connection-like. We derive this model using results from noncompact NHIM theory in a singular perturbation framework. Using a normal form derived from theoretical properties of this reduced-order model, we develop an algorithm that estimates an approximation to the dynamics near a cyclic body shape change (a "gait") directly from observational data of shape and body motion. Our algorithm has applications to the study of optimality of animal gaits, and to hardware-in-the-loop optimization to produce gaits for robots. Finally, we make fundamental contributions to NHIM theory: we prove that the global stable foliation of a NHIM is a $C^0$ disk bundle, and we prove that the dynamics restricted to the stable manifold of a compact inflowing NHIM are globally topologically conjugate to the linearized transverse dynamics at the NHIM restricted to the stable vector bundle. We also give conditions ensuring $C^k$ versions of our results, and we illustrate the theory by giving applications to geometric singular perturbation theory in the case of an attracting critical manifold: we show that the domain of the Fenichel Normal Form can be extended to the entire global stable manifold, and under additional nonresonance assumptions we derive a smooth global linear normal form.PHDElectrical and Computer EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147642/1/kvalheim_1.pd

Data Visualization with Structural Control of Global Cohort and Local Data Neighborhoods

A typical objective of data visualization is to generate low-dimensional plots that maximally convey the information within the data. The visualization output should help the user not only identify the local neighborhood structure of individual samples, but also obtain a global view of the relative positioning and separation between cohorts. Here, we propose a novel visualization framework designed to satisfy these needs. By incorporating additional cohort positioning and discriminative constraints into local neighbor preservation models through the use of computed cohort prototypes, effective control over the arrangements and proximities of data cohorts can be obtained. We introduce various embedding and projection algorithms based on objective functions addressing the different visualization requirements. Their underlying models are optimized effectively using matrix manifold procedures to incorporate the problem constraints. Additionally, to facilitate large-scale applications, a matrix decomposition based model is also proposed to accelerate the computation. The improved capabilities of the new methods are demonstrated using various state-of-the-art dimensionality reduction algorithms. We present many qualitative and quantitative comparisons, on both synthetic problems and real-world tasks of complex text and image data, that show notable improvements over existing techniques

New Directions for Contact Integrators

Contact integrators are a family of geometric numerical schemes which guarantee the conservation of the contact structure. In this work we review the construction of both the variational and Hamiltonian versions of these methods. We illustrate some of the advantages of geometric integration in the dissipative setting by focusing on models inspired by recent studies in celestial mechanics and cosmology.Comment: To appear as Chapter 24 in GSI 2021, Springer LNCS 1282

Notes in Pure Mathematics & Mathematical Structures in Physics

These Notes deal with various areas of mathematics, and seek reciprocal combinations, explore mutual relations, ranging from abstract objects to problems in physics.Comment: Small improvements and addition

Stochastic Filtering on Shape Manifolds

This thesis addresses the problem of learning the dynamics of deforming objects in image time series. In many biomedical imaging and computer vision applications it is important to satisfy certain geometric constraints which traditional time series methods are not capable of handling. We focus on building topology-preserving spatio-temporal stochastic models for shape deformation, which we combine with the observed images to obtain robust object tracking. The shape of the object is modeled as obtained through the action of a group of diffeomorphisms on the initial object boundary. We formulate a state space model for the diffeomorphic deformation of the object, and implement a particle filter on this shape space to estimate the state of the shape in each video frame. We use a practical method for sampling diffeomorphic shapes in which we generate deformations via flows of finitely generated vector fields. Based on the observations and the proposed samples we obtain an approximate estimate for the posterior distribution of the shape. We present the performance of this framework on various image sequences under different scenarios. We extend the random perturbation models to diffusion models on the manifold of planar (discretized) shapes whose drift component represents a trend in the shape deformation. To obtain trends intrinsic to the shape, we define the drift as a gradient of appropriate functions defined over the boundary of the shape. Given a sequence of observations from the path of the suggested stochastic differential equations, we propose a likelihood-ratio-based technique to estimate the missing parameters in the drift terms. We show how to reduce the computational burden and improve the robustness of the estimators by constraining the motion of the shapes to a lower-dimensional submanifold equipped with a sub-Riemannian metric. We further discuss how to apply this methodology to obtain estimates when we have only a limited number of observations