91 research outputs found
Transferability of Graph Neural Networks using Graphon and Sampling Theories
Graph neural networks (GNNs) have become powerful tools for processing
graph-based information in various domains. A desirable property of GNNs is
transferability, where a trained network can swap in information from a
different graph without retraining and retain its accuracy. A recent method of
capturing transferability of GNNs is through the use of graphons, which are
symmetric, measurable functions representing the limit of large dense graphs.
In this work, we contribute to the application of graphons to GNNs by
presenting an explicit two-layer graphon neural network (WNN) architecture. We
prove its ability to approximate bandlimited signals within a specified error
tolerance using a minimal number of network weights. We then leverage this
result, to establish the transferability of an explicit two-layer GNN over all
sufficiently large graphs in a sequence converging to a graphon. Our work
addresses transferability between both deterministic weighted graphs and simple
random graphs and overcomes issues related to the curse of dimensionality that
arise in other GNN results. The proposed WNN and GNN architectures offer
practical solutions for handling graph data of varying sizes while maintaining
performance guarantees without extensive retraining
Non Commutative Convolutional Signal Models in Neural Networks: Stability to Small Deformations
In this paper we discuss the results recently published in~[1] about
algebraic signal models (ASMs) based on non commutative algebras and their use
in convolutional neural networks. Relying on the general tools from algebraic
signal processing (ASP), we study the filtering and stability properties of non
commutative convolutional filters. We show how non commutative filters can be
stable to small perturbations on the space of operators. We also show that
although the spectral components of the Fourier representation in a non
commutative signal model are associated to spaces of dimension larger than one,
there is a trade-off between stability and selectivity similar to that observed
for commutative models. Our results have direct implications for group neural
networks, multigraph neural networks and quaternion neural networks, among
other non commutative architectures. We conclude by corroborating these results
through numerical experiments
Fine-grained Expressivity of Graph Neural Networks
Numerous recent works have analyzed the expressive power of message-passing
graph neural networks (MPNNs), primarily utilizing combinatorial techniques
such as the -dimensional Weisfeiler-Leman test (-WL) for the graph
isomorphism problem. However, the graph isomorphism objective is inherently
binary, not giving insights into the degree of similarity between two given
graphs. This work resolves this issue by considering continuous extensions of
both -WL and MPNNs to graphons. Concretely, we show that the continuous
variant of -WL delivers an accurate topological characterization of the
expressive power of MPNNs on graphons, revealing which graphs these networks
can distinguish and the level of difficulty in separating them. We identify the
finest topology where MPNNs separate points and prove a universal approximation
theorem. Consequently, we provide a theoretical framework for graph and graphon
similarity combining various topological variants of classical
characterizations of the -WL. In particular, we characterize the expressive
power of MPNNs in terms of the tree distance, which is a graph distance based
on the concepts of fractional isomorphisms, and substructure counts via tree
homomorphisms, showing that these concepts have the same expressive power as
the -WL and MPNNs on graphons. Empirically, we validate our theoretical
findings by showing that randomly initialized MPNNs, without training, exhibit
competitive performance compared to their trained counterparts. Moreover, we
evaluate different MPNN architectures based on their ability to preserve graph
distances, highlighting the significance of our continuous -WL test in
understanding MPNNs' expressivity
SC-MAD: Mixtures of Higher-order Networks for Data Augmentation
The myriad complex systems with multiway interactions motivate the extension
of graph-based pairwise connections to higher-order relations. In particular,
the simplicial complex has inspired generalizations of graph neural networks
(GNNs) to simplicial complex-based models. Learning on such systems requires
large amounts of data, which can be expensive or impossible to obtain. We
propose data augmentation of simplicial complexes through both linear and
nonlinear mixup mechanisms that return mixtures of existing labeled samples. In
addition to traditional pairwise mixup, we present a convex clustering mixup
approach for a data-driven relationship among several simplicial complexes. We
theoretically demonstrate that the resultant synthetic simplicial complexes
interpolate among existing data with respect to homomorphism densities. Our
method is demonstrated on both synthetic and real-world datasets for simplicial
complex classification.Comment: 5 pages, 1 figure, 1 tabl
Does Graph Distillation See Like Vision Dataset Counterpart?
Training on large-scale graphs has achieved remarkable results in graph
representation learning, but its cost and storage have attracted increasing
concerns. Existing graph condensation methods primarily focus on optimizing the
feature matrices of condensed graphs while overlooking the impact of the
structure information from the original graphs. To investigate the impact of
the structure information, we conduct analysis from the spectral domain and
empirically identify substantial Laplacian Energy Distribution (LED) shifts in
previous works. Such shifts lead to poor performance in cross-architecture
generalization and specific tasks, including anomaly detection and link
prediction. In this paper, we propose a novel Structure-broadcasting Graph
Dataset Distillation (SGDD) scheme for broadcasting the original structure
information to the generation of the synthetic one, which explicitly prevents
overlooking the original structure information. Theoretically, the synthetic
graphs by SGDD are expected to have smaller LED shifts than previous works,
leading to superior performance in both cross-architecture settings and
specific tasks. We validate the proposed SGDD across 9 datasets and achieve
state-of-the-art results on all of them: for example, on the YelpChi dataset,
our approach maintains 98.6% test accuracy of training on the original graph
dataset with 1,000 times saving on the scale of the graph. Moreover, we
empirically evaluate there exist 17.6% ~ 31.4% reductions in LED shift crossing
9 datasets. Extensive experiments and analysis verify the effectiveness and
necessity of the proposed designs. The code is available in the GitHub
repository: https://github.com/RingBDStack/SGDD.Comment: Accepted by NeurIPS 202
- …