124 research outputs found
On the equivalence between graph isomorphism testing and function approximation with GNNs
Graph neural networks (GNNs) have achieved lots of success on
graph-structured data. In the light of this, there has been increasing interest
in studying their representation power. One line of work focuses on the
universal approximation of permutation-invariant functions by certain classes
of GNNs, and another demonstrates the limitation of GNNs via graph isomorphism
tests.
Our work connects these two perspectives and proves their equivalence. We
further develop a framework of the representation power of GNNs with the
language of sigma-algebra, which incorporates both viewpoints. Using this
framework, we compare the expressive power of different classes of GNNs as well
as other methods on graphs. In particular, we prove that order-2 Graph
G-invariant networks fail to distinguish non-isomorphic regular graphs with the
same degree. We then extend them to a new architecture, Ring-GNNs, which
succeeds on distinguishing these graphs and provides improvements on real-world
social network datasets
Expressive Power of Invariant and Equivariant Graph Neural Networks
Various classes of Graph Neural Networks (GNN) have been proposed and shown
to be successful in a wide range of applications with graph structured data. In
this paper, we propose a theoretical framework able to compare the expressive
power of these GNN architectures. The current universality theorems only apply
to intractable classes of GNNs. Here, we prove the first approximation
guarantees for practical GNNs, paving the way for a better understanding of
their generalization. Our theoretical results are proved for invariant GNNs
computing a graph embedding (permutation of the nodes of the input graph does
not affect the output) and equivariant GNNs computing an embedding of the nodes
(permutation of the input permutes the output). We show that Folklore Graph
Neural Networks (FGNN), which are tensor based GNNs augmented with matrix
multiplication are the most expressive architectures proposed so far for a
given tensor order. We illustrate our results on the Quadratic Assignment
Problem (a NP-Hard combinatorial problem) by showing that FGNNs are able to
learn how to solve the problem, leading to much better average performances
than existing algorithms (based on spectral, SDP or other GNNs architectures).
On a practical side, we also implement masked tensors to handle batches of
graphs of varying sizes.Comment: Appears in: Proceedings of the 9th International Conference on
Learning Representations, ICLR 2021. 39 page
The Expressive Power of Graph Neural Networks: A Survey
Graph neural networks (GNNs) are effective machine learning models for many
graph-related applications. Despite their empirical success, many research
efforts focus on the theoretical limitations of GNNs, i.e., the GNNs expressive
power. Early works in this domain mainly focus on studying the graph
isomorphism recognition ability of GNNs, and recent works try to leverage the
properties such as subgraph counting and connectivity learning to characterize
the expressive power of GNNs, which are more practical and closer to
real-world. However, no survey papers and open-source repositories
comprehensively summarize and discuss models in this important direction. To
fill the gap, we conduct a first survey for models for enhancing expressive
power under different forms of definition. Concretely, the models are reviewed
based on three categories, i.e., Graph feature enhancement, Graph topology
enhancement, and GNNs architecture enhancement
Neural function approximation on graphs: shape modelling, graph discrimination & compression
Graphs serve as a versatile mathematical abstraction of real-world phenomena in numerous scientific disciplines. This thesis is part of the Geometric Deep Learning subject area, a family of learning paradigms, that capitalise on the increasing volume of non-Euclidean data so as to solve real-world tasks in a data-driven manner. In particular, we focus on the topic of graph function approximation using neural networks, which lies at the heart of many relevant methods. In the first part of the thesis, we contribute to the understanding and design of Graph Neural Networks (GNNs). Initially, we investigate the problem of learning on signals supported on a fixed graph. We show that treating graph signals as general graph spaces is restrictive and conventional GNNs have limited expressivity. Instead, we expose a more enlightening perspective by drawing parallels between graph signals and signals on Euclidean grids, such as images and audio. Accordingly, we propose a permutation-sensitive GNN based on an operator analogous to shifts in grids and instantiate it on 3D meshes for shape modelling (Spiral Convolutions). Following, we focus on learning on general graph spaces and in particular on functions that are invariant to graph isomorphism. We identify a fundamental trade-off between invariance, expressivity and computational complexity, which we address with a symmetry-breaking mechanism based on substructure encodings (Graph Substructure Networks). Substructures are shown to be a powerful tool that provably improves expressivity while controlling computational complexity, and a useful inductive bias in network science and chemistry. In the second part of the thesis, we discuss the problem of graph compression, where we analyse the information-theoretic principles and the connections with graph generative models. We show that another inevitable trade-off surfaces, now between computational complexity and compression quality, due to graph isomorphism. We propose a substructure-based dictionary coder - Partition and Code (PnC) - with theoretical guarantees that can be adapted to different graph distributions by estimating its parameters from observations. Additionally, contrary to the majority of neural compressors, PnC is parameter and sample efficient and is therefore of wide practical relevance. Finally, within this framework, substructures are further illustrated as a decisive archetype for learning problems on graph spaces.Open Acces
Weisfeiler--Lehman goes Dynamic: An Analysis of the Expressive Power of Graph Neural Networks for Attributed and Dynamic Graphs
Graph Neural Networks (GNNs) are a large class of relational models for graph
processing. Recent theoretical studies on the expressive power of GNNs have
focused on two issues. On the one hand, it has been proven that GNNs are as
powerful as the Weisfeiler-Lehman test (1-WL) in their ability to distinguish
graphs. Moreover, it has been shown that the equivalence enforced by 1-WL
equals unfolding equivalence. On the other hand, GNNs turned out to be
universal approximators on graphs modulo the constraints enforced by
1-WL/unfolding equivalence. However, these results only apply to Static
Undirected Homogeneous Graphs with node attributes. In contrast, real-life
applications often involve a variety of graph properties, such as, e.g.,
dynamics or node and edge attributes. In this paper, we conduct a theoretical
analysis of the expressive power of GNNs for these two graph types that are
particularly of interest. Dynamic graphs are widely used in modern
applications, and its theoretical analysis requires new approaches. The
attributed type acts as a standard form for all graph types since it has been
shown that all graph types can be transformed without loss to Static Undirected
Homogeneous Graphs with attributes on nodes and edges (SAUHG). The study
considers generic GNN models and proposes appropriate 1-WL tests for those
domains. Then, the results on the expressive power of GNNs are extended by
proving that GNNs have the same capability as the 1-WL test in distinguishing
dynamic and attributed graphs, the 1-WL equivalence equals unfolding
equivalence and that GNNs are universal approximators modulo 1-WL/unfolding
equivalence. Moreover, the proof of the approximation capability holds for
SAUHGs, which include most of those used in practical applications, and it is
constructive in nature allowing to deduce hints on the architecture of GNNs
that can achieve the desired accuracy
- …