24 research outputs found
Intrinsic Universal Measurements of Non-linear Embeddings
A basic problem in machine learning is to find a mapping from a low
dimensional latent space to a high dimensional observation space. Equipped with
the representation power of non-linearity, a learner can easily find a mapping
which perfectly fits all the observations. However such a mapping is often not
considered as good as it is not simple enough and over-fits. How to define
simplicity? This paper tries to make such a formal definition of the amount of
information imposed by a non-linear mapping. This definition is based on
information geometry and is independent of observations, nor specific
parametrizations. We prove these basic properties and discuss relationships
with parametric and non-parametric embeddings.Comment: work in progres
Beyond Sentiment: The Manifold of Human Emotions
Sentiment analysis predicts the presence of positive or negative emotions in
a text document. In this paper we consider higher dimensional extensions of the
sentiment concept, which represent a richer set of human emotions. Our approach
goes beyond previous work in that our model contains a continuous manifold
rather than a finite set of human emotions. We investigate the resulting model,
compare it to psychological observations, and explore its predictive
capabilities. Besides obtaining significant improvements over a baseline
without manifold, we are also able to visualize different notions of positive
sentiment in different domains.Comment: 15 pages, 7 figure
Geometrically Enriched Latent Spaces
A common assumption in generative models is that the generator immerses the
latent space into a Euclidean ambient space. Instead, we consider the ambient
space to be a Riemannian manifold, which allows for encoding domain knowledge
through the associated Riemannian metric. Shortest paths can then be defined
accordingly in the latent space to both follow the learned manifold and respect
the ambient geometry. Through careful design of the ambient metric we can
ensure that shortest paths are well-behaved even for deterministic generators
that otherwise would exhibit a misleading bias. Experimentally we show that our
approach improves interpretability of learned representations both using
stochastic and deterministic generators
Regression-Based Elastic Metric Learning on Shape Spaces of Elastic Curves
We propose a metric learning paradigm, Regression-based Elastic Metric
Learning (REML), which optimizes the elastic metric for geodesic regression on
the manifold of discrete curves. Geodesic regression is most accurate when the
chosen metric models the data trajectory close to a geodesic on the discrete
curve manifold. When tested on cell shape trajectories, regression with REML's
learned metric has better predictive power than with the conventionally used
square-root-velocity (SRV) metric.Comment: 4 pages, 2 figures, derivations in appendi
Parametric information geometry with the package Geomstats
We introduce the information geometry module of the Python package Geomstats.
The module first implements Fisher-Rao Riemannian manifolds of widely used
parametric families of probability distributions, such as normal, gamma, beta,
Dirichlet distributions, and more. The module further gives the Fisher-Rao
Riemannian geometry of any parametric family of distributions of interest,
given a parameterized probability density function as input. The implemented
Riemannian geometry tools allow users to compare, average, interpolate between
distributions inside a given family. Importantly, such capabilities open the
door to statistics and machine learning on probability distributions. We
present the object-oriented implementation of the module along with
illustrative examples and show how it can be used to perform learning on
manifolds of parametric probability distributions
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold
Although Deep Learning (DL) has achieved success in complex Artificial
Intelligence (AI) tasks, it suffers from various notorious problems (e.g.,
feature redundancy, and vanishing or exploding gradients), since updating
parameters in Euclidean space cannot fully exploit the geometric structure of
the solution space. As a promising alternative solution, Riemannian-based DL
uses geometric optimization to update parameters on Riemannian manifolds and
can leverage the underlying geometric information. Accordingly, this article
presents a comprehensive survey of applying geometric optimization in DL. At
first, this article introduces the basic procedure of the geometric
optimization, including various geometric optimizers and some concepts of
Riemannian manifold. Subsequently, this article investigates the application of
geometric optimization in different DL networks in various AI tasks, e.g.,
convolution neural network, recurrent neural network, transfer learning, and
optimal transport. Additionally, typical public toolboxes that implement
optimization on manifold are also discussed. Finally, this article makes a
performance comparison between different deep geometric optimization methods
under image recognition scenarios.Comment: 41 page