2 research outputs found
Adaptive Universal Generalized PageRank Graph Neural Network
In many important graph data processing applications the acquired information
includes both node features and observations of the graph topology. Graph
neural networks (GNNs) are designed to exploit both sources of evidence but
they do not optimally trade-off their utility and integrate them in a manner
that is also universal. Here, universality refers to independence on homophily
or heterophily graph assumptions. We address these issues by introducing a new
Generalized PageRank (GPR) GNN architecture that adaptively learns the GPR
weights so as to jointly optimize node feature and topological information
extraction, regardless of the extent to which the node labels are homophilic or
heterophilic. Learned GPR weights automatically adjust to the node label
pattern, irrelevant on the type of initialization, and thereby guarantee
excellent learning performance for label patterns that are usually hard to
handle. Furthermore, they allow one to avoid feature over-smoothing, a process
which renders feature information nondiscriminative, without requiring the
network to be shallow. Our accompanying theoretical analysis of the GPR-GNN
method is facilitated by novel synthetic benchmark datasets generated by the
so-called contextual stochastic block model. We also compare the performance of
our GNN architecture with that of several state-of-the-art GNNs on the problem
of node-classification, using well-known benchmark homophilic and heterophilic
datasets. The results demonstrate that GPR-GNN offers significant performance
improvement compared to existing techniques on both synthetic and benchmark
data.Comment: ICLR 202
Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning
Learning representations of sets of nodes in a graph is crucial for
applications ranging from node-role discovery to link prediction and molecule
classification. Graph Neural Networks (GNNs) have achieved great success in
graph representation learning. However, expressive power of GNNs is limited by
the 1-Weisfeiler-Lehman (WL) test and thus GNNs generate identical
representations for graph substructures that may in fact be very different.
More powerful GNNs, proposed recently by mimicking higher-order-WL tests, only
focus on representing entire graphs and they are computationally inefficient as
they cannot utilize sparsity of the underlying graph. Here we propose and
mathematically analyze a general class of structure-related features, termed
Distance Encoding (DE). DE assists GNNs in representing any set of nodes, while
providing strictly more expressive power than the 1-WL test. DE captures the
distance between the node set whose representation is to be learned and each
node in the graph. To capture the distance DE can apply various graph-distance
measures such as shortest path distance or generalized PageRank scores. We
propose two ways for GNNs to use DEs (1) as extra node features, and (2) as
controllers of message aggregation in GNNs. Both approaches can utilize the
sparse structure of the underlying graph, which leads to computational
efficiency and scalability. We also prove that DE can distinguish node sets
embedded in almost all regular graphs where traditional GNNs always fail. We
evaluate DE on three tasks over six real networks: structural role prediction,
link prediction, and triangle prediction. Results show that our models
outperform GNNs without DE by up-to 15\% in accuracy and AUROC. Furthermore,
our models also significantly outperform other state-of-the-art methods
especially designed for the above tasks.Comment: NeurIPS 202