9,997 research outputs found
Geometric Graph Filters and Neural Networks: Limit Properties and Discriminability Trade-offs
This paper studies the relationship between a graph neural network (GNN) and
a manifold neural network (MNN) when the graph is constructed from a set of
points sampled from the manifold, thus encoding geometric information. We
consider convolutional MNNs and GNNs where the manifold and the graph
convolutions are respectively defined in terms of the Laplace-Beltrami operator
and the graph Laplacian. Using the appropriate kernels, we analyze both dense
and moderately sparse graphs. We prove non-asymptotic error bounds showing that
convolutional filters and neural networks on these graphs converge to
convolutional filters and neural networks on the continuous manifold. As a
byproduct of this analysis, we observe an important trade-off between the
discriminability of graph filters and their ability to approximate the desired
behavior of manifold filters. We then discuss how this trade-off is ameliorated
in neural networks due to the frequency mixing property of nonlinearities. We
further derive a transferability corollary for geometric graphs sampled from
the same manifold. We validate our results numerically on a navigation control
problem and a point cloud classification task.Comment: 16 pages, 6 figures, 3 table
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Recent advancements in deep neural networks for graph-structured data have
led to state-of-the-art performance on recommender system benchmarks. However,
making these methods practical and scalable to web-scale recommendation tasks
with billions of items and hundreds of millions of users remains a challenge.
Here we describe a large-scale deep recommendation engine that we developed and
deployed at Pinterest. We develop a data-efficient Graph Convolutional Network
(GCN) algorithm PinSage, which combines efficient random walks and graph
convolutions to generate embeddings of nodes (i.e., items) that incorporate
both graph structure as well as node feature information. Compared to prior GCN
approaches, we develop a novel method based on highly efficient random walks to
structure the convolutions and design a novel training strategy that relies on
harder-and-harder training examples to improve robustness and convergence of
the model. We also develop an efficient MapReduce model inference algorithm to
generate embeddings using a trained model. We deploy PinSage at Pinterest and
train it on 7.5 billion examples on a graph with 3 billion nodes representing
pins and boards, and 18 billion edges. According to offline metrics, user
studies and A/B tests, PinSage generates higher-quality recommendations than
comparable deep learning and graph-based alternatives. To our knowledge, this
is the largest application of deep graph embeddings to date and paves the way
for a new generation of web-scale recommender systems based on graph
convolutional architectures.Comment: KDD 201
Increase and Conquer: Training Graph Neural Networks on Growing Graphs
Graph neural networks (GNNs) use graph convolutions to exploit network
invariances and learn meaningful features from network data. However, on
large-scale graphs convolutions incur in high computational cost, leading to
scalability limitations. Leveraging the graphon -- the limit object of a graph
-- in this paper we consider the problem of learning a graphon neural network
(WNN) -- the limit object of a GNN -- by training GNNs on graphs sampled
Bernoulli from the graphon. Under smoothness conditions, we show that: (i) the
expected distance between the learning steps on the GNN and on the WNN
decreases asymptotically with the size of the graph, and (ii) when training on
a sequence of growing graphs, gradient descent follows the learning direction
of the WNN. Inspired by these results, we propose a novel algorithm to learn
GNNs on large-scale graphs that, starting from a moderate number of nodes,
successively increases the size of the graph during training. This algorithm is
benchmarked on both a recommendation system and a decentralized control problem
where it is shown to retain comparable performance, to its large-scale
counterpart, at a reduced computational cost
- …