2,138 research outputs found
Neural Embeddings of Graphs in Hyperbolic Space
Neural embeddings have been used with great success in Natural Language
Processing (NLP). They provide compact representations that encapsulate word
similarity and attain state-of-the-art performance in a range of linguistic
tasks. The success of neural embeddings has prompted significant amounts of
research into applications in domains other than language. One such domain is
graph-structured data, where embeddings of vertices can be learned that
encapsulate vertex similarity and improve performance on tasks including edge
prediction and vertex labelling. For both NLP and graph based tasks, embeddings
have been learned in high-dimensional Euclidean spaces. However, recent work
has shown that the appropriate isometric space for embedding complex networks
is not the flat Euclidean space, but negatively curved, hyperbolic space. We
present a new concept that exploits these recent insights and propose learning
neural embeddings of graphs in hyperbolic space. We provide experimental
evidence that embedding graphs in their natural geometry significantly improves
performance on downstream tasks for several real-world public datasets.Comment: 7 pages, 5 figure
Hyperbolic Interaction Model For Hierarchical Multi-Label Classification
Different from the traditional classification tasks which assume mutual
exclusion of labels, hierarchical multi-label classification (HMLC) aims to
assign multiple labels to every instance with the labels organized under
hierarchical relations. Besides the labels, since linguistic ontologies are
intrinsic hierarchies, the conceptual relations between words can also form
hierarchical structures. Thus it can be a challenge to learn mappings from word
hierarchies to label hierarchies. We propose to model the word and label
hierarchies by embedding them jointly in the hyperbolic space. The main reason
is that the tree-likeness of the hyperbolic space matches the complexity of
symbolic data with hierarchical structures. A new Hyperbolic Interaction Model
(HyperIM) is designed to learn the label-aware document representations and
make predictions for HMLC. Extensive experiments are conducted on three
benchmark datasets. The results have demonstrated that the new model can
realistically capture the complex data structures and further improve the
performance for HMLC comparing with the state-of-the-art methods. To facilitate
future research, our code is publicly available
HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization
Supervision for metric learning has long been given in the form of
equivalence between human-labeled classes. Although this type of supervision
has been a basis of metric learning for decades, we argue that it hinders
further advances of the field. In this regard, we propose a new regularization
method, dubbed HIER, to discover the latent semantic hierarchy of training
data, and to deploy the hierarchy to provide richer and more fine-grained
supervision than inter-class separability induced by common metric learning
losses. HIER achieved this goal with no annotation for the semantic hierarchy
but by learning hierarchical proxies in hyperbolic spaces. The hierarchical
proxies are learnable parameters, and each of them is trained to serve as an
ancestor of a group of data or other proxies to approximate the semantic
hierarchy among them. HIER deals with the proxies along with data in hyperbolic
space since geometric properties of the space are well-suited to represent
their hierarchical structure. The efficacy of HIER was evaluated on four
standard benchmarks, where it consistently improved performance of conventional
methods when integrated with them, and consequently achieved the best records,
surpassing even the existing hyperbolic metric learning technique, in almost
all settings
Hyperbolic Geometry in Computer Vision: A Survey
Hyperbolic geometry, a Riemannian manifold endowed with constant sectional
negative curvature, has been considered an alternative embedding space in many
learning scenarios, \eg, natural language processing, graph learning, \etc, as
a result of its intriguing property of encoding the data's hierarchical
structure (like irregular graph or tree-likeness data). Recent studies prove
that such data hierarchy also exists in the visual dataset, and investigate the
successful practice of hyperbolic geometry in the computer vision (CV) regime,
ranging from the classical image classification to advanced model adaptation
learning. This paper presents the first and most up-to-date literature review
of hyperbolic spaces for CV applications. To this end, we first introduce the
background of hyperbolic geometry, followed by a comprehensive investigation of
algorithms, with geometric prior of hyperbolic space, in the context of visual
applications. We also conclude this manuscript and identify possible future
directions.Comment: First survey paper for the hyperbolic geometry in CV application
Hyperbolic Deep Neural Networks: A Survey
Recently, there has been a rising surge of momentum for deep representation
learning in hyperbolic spaces due to theirhigh capacity of modeling data like
knowledge graphs or synonym hierarchies, possessing hierarchical structure. We
refer to the model as hyperbolic deep neural network in this paper. Such a
hyperbolic neural architecture potentially leads to drastically compact model
withmuch more physical interpretability than its counterpart in Euclidean
space. To stimulate future research, this paper presents acoherent and
comprehensive review of the literature around the neural components in the
construction of hyperbolic deep neuralnetworks, as well as the generalization
of the leading deep approaches to the Hyperbolic space. It also presents
current applicationsaround various machine learning tasks on several publicly
available datasets, together with insightful observations and identifying
openquestions and promising future directions
- …