32 research outputs found
Over-Squashing in Riemannian Graph Neural Networks
Most graph neural networks (GNNs) are prone to the phenomenon of
over-squashing in which node features become insensitive to information from
distant nodes in the graph. Recent works have shown that the topology of the
graph has the greatest impact on over-squashing, suggesting graph rewiring
approaches as a suitable solution. In this work, we explore whether
over-squashing can be mitigated through the embedding space of the GNN. In
particular, we consider the generalization of Hyperbolic GNNs (HGNNs) to
Riemannian manifolds of variable curvature in which the geometry of the
embedding space is faithful to the graph's topology. We derive bounds on the
sensitivity of the node features in these Riemannian GNNs as the number of
layers increases, which yield promising theoretical and empirical results for
alleviating over-squashing in graphs with negative curvature
FMGNN: Fused Manifold Graph Neural Network
Graph representation learning has been widely studied and demonstrated
effectiveness in various graph tasks. Most existing works embed graph data in
the Euclidean space, while recent works extend the embedding models to
hyperbolic or spherical spaces to achieve better performance on graphs with
complex structures, such as hierarchical or ring structures. Fusing the
embedding from different manifolds can further take advantage of the embedding
capabilities over different graph structures. However, existing embedding
fusion methods mostly focus on concatenating or summing up the output
embeddings, without considering interacting and aligning the embeddings of the
same vertices on different manifolds, which can lead to distortion and
impression in the final fusion results. Besides, it is also challenging to fuse
the embeddings of the same vertices from different coordinate systems. In face
of these challenges, we propose the Fused Manifold Graph Neural Network
(FMGNN), a novel GNN architecture that embeds graphs into different Riemannian
manifolds with interaction and alignment among these manifolds during training
and fuses the vertex embeddings through the distances on different manifolds
between vertices and selected landmarks, geometric coresets. Our experiments
demonstrate that FMGNN yields superior performance over strong baselines on the
benchmarks of node classification and link prediction tasks