Size Generalization of Graph Neural Networks on Biological Data: Insights and Practices from the Spectral Perspective

We investigate size-induced distribution shifts in graphs and assess their impact on the ability of graph neural networks (GNNs) to generalize to larger graphs relative to the training data. Existing literature presents conflicting conclusions on GNNs' size generalizability, primarily due to disparities in application domains and underlying assumptions concerning size-induced distribution shifts. Motivated by this, we take a data-driven approach: we focus on real biological datasets and seek to characterize the types of size-induced distribution shifts. Diverging from prior approaches, we adopt a spectral perspective and identify that spectrum differences induced by size are related to differences in subgraph patterns (e.g., average cycle lengths). While previous studies have identified that the inability of GNNs in capturing subgraph information negatively impacts their in-distribution generalization, our findings further show that this decline is more pronounced when evaluating on larger test graphs not encountered during training. Based on these spectral insights, we introduce a simple yet effective model-agnostic strategy, which makes GNNs aware of these important subgraph patterns to enhance their size generalizability. Our empirical results reveal that our proposed size-insensitive attention strategy substantially enhances graph classification performance on large test graphs, which are 2-10 times larger than the training graphs, resulting in an improvement in F1 scores by up to 8%.Comment: 21 pages, including appendi

Topology Discovery of Sparse Random Graphs With Few Participants

We consider the task of topology discovery of sparse random graphs using end-to-end random measurements (e.g., delay) between a subset of nodes, referred to as the participants. The rest of the nodes are hidden, and do not provide any information for topology discovery. We consider topology discovery under two routing models: (a) the participants exchange messages along the shortest paths and obtain end-to-end measurements, and (b) additionally, the participants exchange messages along the second shortest path. For scenario (a), our proposed algorithm results in a sub-linear edit-distance guarantee using a sub-linear number of uniformly selected participants. For scenario (b), we obtain a much stronger result, and show that we can achieve consistent reconstruction when a sub-linear number of uniformly selected nodes participate. This implies that accurate discovery of sparse random graphs is tractable using an extremely small number of participants. We finally obtain a lower bound on the number of participants required by any algorithm to reconstruct the original random graph up to a given edit distance. We also demonstrate that while consistent discovery is tractable for sparse random graphs using a small number of participants, in general, there are graphs which cannot be discovered by any algorithm even with a significant number of participants, and with the availability of end-to-end information along all the paths between the participants.Comment: A shorter version appears in ACM SIGMETRICS 2011. This version is scheduled to appear in J. on Random Structures and Algorithm

Topological Resonances in Scattering on Networks (Graphs)

We report on a hitherto unnoticed type of resonances occurring in scattering from networks (quantum graphs) which are due to the complex connectivity of the graph - its topology. We consider generic open graphs and show that any cycle leads to narrow resonances which do not fit in any of the prominent paradigms for narrow resonances (classical barriers, localization due to disorder, chaotic scattering). We call these resonances `topological' to emphasize their origin in the non-trivial connectivity. Topological resonances have a clear and unique signature which is apparent in the statistics of the resonance parameters (such as e.g., the width, the delay time or the wave-function intensity in the graph). We discuss this phenomenon by providing analytical arguments supported by numerical simulation, and identify the features of the above distributions which depend on genuine topological quantities such as the length of the shortest cycle (girth). These signatures cannot be explained using any of the other paradigms for narrow resonances. Finally, we propose an experimental setting where the topological resonances could be demonstrated, and study the stability of the relevant distribution functions to moderate dissipation

Distribution of shortest cycle lengths in random networks

We present analytical results for the distribution of shortest cycle lengths (DSCL) in random networks. The approach is based on the relation between the DSCL and the distribution of shortest path lengths (DSPL). We apply this approach to configuration model networks, for which analytical results for the DSPL were obtained before. We first calculate the fraction of nodes in the network which reside on at least one cycle. Conditioning on being on a cycle, we provide the DSCL over ensembles of configuration model networks with degree distributions which follow a Poisson distribution (Erdos-R\'enyi network), degenerate distribution (random regular graph) and a power-law distribution (scale-free network). The mean and variance of the DSCL are calculated. The analytical results are found to be in very good agreement with the results of computer simulations.Comment: 44 pages, 11 figure