8,992 research outputs found
Neural Embeddings of Graphs in Hyperbolic Space
Neural embeddings have been used with great success in Natural Language
Processing (NLP). They provide compact representations that encapsulate word
similarity and attain state-of-the-art performance in a range of linguistic
tasks. The success of neural embeddings has prompted significant amounts of
research into applications in domains other than language. One such domain is
graph-structured data, where embeddings of vertices can be learned that
encapsulate vertex similarity and improve performance on tasks including edge
prediction and vertex labelling. For both NLP and graph based tasks, embeddings
have been learned in high-dimensional Euclidean spaces. However, recent work
has shown that the appropriate isometric space for embedding complex networks
is not the flat Euclidean space, but negatively curved, hyperbolic space. We
present a new concept that exploits these recent insights and propose learning
neural embeddings of graphs in hyperbolic space. We provide experimental
evidence that embedding graphs in their natural geometry significantly improves
performance on downstream tasks for several real-world public datasets.Comment: 7 pages, 5 figure
Computationally Tractable Riemannian Manifolds for Graph Embeddings
Representing graphs as sets of node embeddings in certain curved Riemannian
manifolds has recently gained momentum in machine learning due to their
desirable geometric inductive biases, e.g., hierarchical structures benefit
from hyperbolic geometry. However, going beyond embedding spaces of constant
sectional curvature, while potentially more representationally powerful, proves
to be challenging as one can easily lose the appeal of computationally
tractable tools such as geodesic distances or Riemannian gradients. Here, we
explore computationally efficient matrix manifolds, showcasing how to learn and
optimize graph embeddings in these Riemannian spaces. Empirically, we
demonstrate consistent improvements over Euclidean geometry while often
outperforming hyperbolic and elliptical embeddings based on various metrics
that capture different graph properties. Our results serve as new evidence for
the benefits of non-Euclidean embeddings in machine learning pipelines.Comment: Submitted to the Thirty-fourth Conference on Neural Information
Processing System
Monotone Maps, Sphericity and Bounded Second Eigenvalue
We consider {\em monotone} embeddings of a finite metric space into low
dimensional normed space. That is, embeddings that respect the order among the
distances in the original space. Our main interest is in embeddings into
Euclidean spaces. We observe that any metric on points can be embedded into
, while, (in a sense to be made precise later), for almost every
-point metric space, every monotone map must be into a space of dimension
.
It becomes natural, then, to seek explicit constructions of metric spaces
that cannot be monotonically embedded into spaces of sublinear dimension. To
this end, we employ known results on {\em sphericity} of graphs, which suggest
one example of such a metric space - that defined by a complete bipartitegraph.
We prove that an -regular graph of order , with bounded diameter
has sphericity , where is the second
largest eigenvalue of the adjacency matrix of the graph, and 0 < \delta \leq
\half is constant. We also show that while random graphs have linear
sphericity, there are {\em quasi-random} graphs of logarithmic sphericity.
For the above bound to be linear, must be constant. We show that
if the second eigenvalue of an -regular graph is bounded by a constant,
then the graph is close to being complete bipartite. Namely, its adjacency
matrix differs from that of a complete bipartite graph in only
entries. Furthermore, for any 0 < \delta < \half, and , there are
only finitely many -regular graphs with second eigenvalue at most
Learning Neural Graph Representations in Non-Euclidean Geometries
The success of Deep Learning methods is heavily dependent on the choice of the data representation. For that reason, much of the actual effort goes into Representation Learning, which seeks to design preprocessing pipelines and data transformations that can support effective learning algorithms. The aim of Representation Learning is to facilitate the task of extracting useful information for classifiers and other predictor models. In this regard, graphs arise as a convenient data structure that serves as an intermediary representation in a wide range of problems. The predominant approach to work with graphs has been to embed them in an Euclidean space, due to the power and simplicity of this geometry. Nevertheless, data in many domains exhibit non-Euclidean features, making embeddings into Riemannian manifolds with a richer structure necessary. The choice of a metric space where to embed the data imposes a geometric inductive bias, with a direct impact on the performance of the models.
This thesis is about learning neural graph representations in non-Euclidean geometries and showcasing their applicability in different downstream tasks. We introduce a toolkit formed by different graph metrics with the goal of characterizing the topology of the data. In that way, we can choose a suitable target embedding space aligned to the shape of the dataset. By virtue of the geometric inductive bias provided by the structure of the non-Euclidean manifolds, neural models can achieve higher performances with a reduced parameter footprint.
As a first step, we study graphs with hierarchical structures. We develop different techniques to derive hierarchical graphs from large label inventories. Noticing the capacity of hyperbolic spaces to represent tree-like arrangements, we incorporate this information into an NLP model through hyperbolic graph embeddings and showcase the higher performance that they enable.
Second, we tackle the question of how to learn hierarchical representations suited for different downstream tasks. We introduce a model that jointly learns task-specific graph embeddings from a label inventory and performs classification in hyperbolic space. The model achieves state-of-the-art results on very fine-grained labels, with a remarkable reduction of the parameter size.
Next, we move to matrix manifolds to work on graphs with diverse structures and properties. We propose a general framework to implement the mathematical tools required to learn graph embeddings on symmetric spaces. These spaces are of particular interest given that they have a compound geometry that simultaneously contains Euclidean as well as hyperbolic subspaces, allowing them to automatically adapt to dissimilar features in the graph. We demonstrate a concrete implementation of the framework on Siegel spaces, showcasing their versatility on different tasks.
Finally, we focus on multi-relational graphs. We devise the means to translate Euclidean and hyperbolic multi-relational graph embedding models into the space of symmetric positive definite (SPD) matrices. To do so we develop gyrocalculus in this geometry and integrate it with the aforementioned framework
Change Detection in Graph Streams by Learning Graph Embeddings on Constant-Curvature Manifolds
The space of graphs is often characterised by a non-trivial geometry, which
complicates learning and inference in practical applications. A common approach
is to use embedding techniques to represent graphs as points in a conventional
Euclidean space, but non-Euclidean spaces have often been shown to be better
suited for embedding graphs. Among these, constant-curvature Riemannian
manifolds (CCMs) offer embedding spaces suitable for studying the statistical
properties of a graph distribution, as they provide ways to easily compute
metric geodesic distances. In this paper, we focus on the problem of detecting
changes in stationarity in a stream of attributed graphs. To this end, we
introduce a novel change detection framework based on neural networks and CCMs,
that takes into account the non-Euclidean nature of graphs. Our contribution in
this work is twofold. First, via a novel approach based on adversarial
learning, we compute graph embeddings by training an autoencoder to represent
graphs on CCMs. Second, we introduce two novel change detection tests operating
on CCMs. We perform experiments on synthetic data, as well as two real-world
application scenarios: the detection of epileptic seizures using functional
connectivity brain networks, and the detection of hostility between two
subjects, using human skeletal graphs. Results show that the proposed methods
are able to detect even small changes in a graph-generating process,
consistently outperforming approaches based on Euclidean embeddings.Comment: 14 pages, 8 figure
- …