33,604 research outputs found
Learning Edge Representations via Low-Rank Asymmetric Projections
We propose a new method for embedding graphs while preserving directed edge
information. Learning such continuous-space vector representations (or
embeddings) of nodes in a graph is an important first step for using network
information (from social networks, user-item graphs, knowledge bases, etc.) in
many machine learning tasks.
Unlike previous work, we (1) explicitly model an edge as a function of node
embeddings, and we (2) propose a novel objective, the "graph likelihood", which
contrasts information from sampled random walks with non-existent edges.
Individually, both of these contributions improve the learned representations,
especially when there are memory constraints on the total size of the
embeddings. When combined, our contributions enable us to significantly improve
the state-of-the-art by learning more concise representations that better
preserve the graph structure.
We evaluate our method on a variety of link-prediction task including social
networks, collaboration networks, and protein interactions, showing that our
proposed method learn representations with error reductions of up to 76% and
55%, on directed and undirected graphs. In addition, we show that the
representations learned by our method are quite space efficient, producing
embeddings which have higher structure-preserving accuracy but are 10 times
smaller
Predefined Sparseness in Recurrent Sequence Models
Inducing sparseness while training neural networks has been shown to yield
models with a lower memory footprint but similar effectiveness to dense models.
However, sparseness is typically induced starting from a dense model, and thus
this advantage does not hold during training. We propose techniques to enforce
sparseness upfront in recurrent sequence models for NLP applications, to also
benefit training. First, in language modeling, we show how to increase hidden
state sizes in recurrent layers without increasing the number of parameters,
leading to more expressive models. Second, for sequence labeling, we show that
word embeddings with predefined sparseness lead to similar performance as dense
embeddings, at a fraction of the number of trainable parameters.Comment: the SIGNLL Conference on Computational Natural Language Learning
(CoNLL, 2018
MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction
Autonomous vehicles are expected to drive in complex scenarios with several
independent non cooperating agents. Path planning for safely navigating in such
environments can not just rely on perceiving present location and motion of
other agents. It requires instead to predict such variables in a far enough
future. In this paper we address the problem of multimodal trajectory
prediction exploiting a Memory Augmented Neural Network. Our method learns past
and future trajectory embeddings using recurrent neural networks and exploits
an associative external memory to store and retrieve such embeddings.
Trajectory prediction is then performed by decoding in-memory future encodings
conditioned with the observed past. We incorporate scene knowledge in the
decoding state by learning a CNN on top of semantic scene maps. Memory growth
is limited by learning a writing controller based on the predictive capability
of existing embeddings. We show that our method is able to natively perform
multi-modal trajectory prediction obtaining state-of-the art results on three
datasets. Moreover, thanks to the non-parametric nature of the memory module,
we show how once trained our system can continuously improve by ingesting novel
patterns.Comment: Accepted at CVPR2
- …