26 research outputs found
WL meet VC
Recently, many works studied the expressive power of graph neural networks
(GNNs) by linking it to the -dimensional Weisfeiler--Leman algorithm
(). Here, the is a well-studied
heuristic for the graph isomorphism problem, which iteratively colors or
partitions a graph's vertex set. While this connection has led to significant
advances in understanding and enhancing GNNs' expressive power, it does not
provide insights into their generalization performance, i.e., their ability to
make meaningful predictions beyond the training set. In this paper, we study
GNNs' generalization ability through the lens of Vapnik--Chervonenkis (VC)
dimension theory in two settings, focusing on graph-level predictions. First,
when no upper bound on the graphs' order is known, we show that the bitlength
of GNNs' weights tightly bounds their VC dimension. Further, we derive an upper
bound for GNNs' VC dimension using the number of colors produced by the
. Secondly, when an upper bound on the graphs' order is
known, we show a tight connection between the number of graphs distinguishable
by the and GNNs' VC dimension. Our empirical study
confirms the validity of our theoretical findings.Comment: arXiv admin note: text overlap with arXiv:2206.1116
Logical Equivalences, Homomorphism Indistinguishability, and Forbidden Minors
Two graphs and are homomorphism indistinguishable over a class of
graphs if for all graphs the number of
homomorphisms from to is equal to the number of homomorphisms from
to . Many natural equivalence relations comparing graphs such as (quantum)
isomorphism, spectral, and logical equivalences can be characterised as
homomorphism indistinguishability relations over certain graph classes.
Abstracting from the wealth of such instances, we show in this paper that
equivalences w.r.t. any self-complementarity logic admitting a characterisation
as homomorphism indistinguishability relation can be characterised by
homomorphism indistinguishability over a minor-closed graph class.
Self-complementarity is a mild property satisfied by most well-studied logics.
This result follows from a correspondence between closure properties of a graph
class and preservation properties of its homomorphism indistinguishability
relation.
Furthermore, we classify all graph classes which are in a sense finite
(essentially profinite) and satisfy the maximality condition of being
homomorphism distinguishing closed, i.e. adding any graph to the class strictly
refines its homomorphism indistinguishability relation. Thereby, we answer
various question raised by Roberson (2022) on general properties of the
homomorphism distinguishing closure.Comment: 26 pages, 1 figure, 1 tabl
Improving Expressivity of Graph Neural Networks using Localization
In this paper, we propose localized versions of Weisfeiler-Leman (WL)
algorithms in an effort to both increase the expressivity, as well as decrease
the computational overhead. We focus on the specific problem of subgraph
counting and give localized versions of WL for any . We analyze the
power of Local WL and prove that it is more expressive than WL and at
most as expressive as WL. We give a characterization of patterns whose
count as a subgraph and induced subgraph are invariant if two graphs are Local
WL equivalent. We also introduce two variants of WL: Layer WL and
recursive WL. These methods are more time and space efficient than applying
WL on the whole graph. We also propose a fragmentation technique that
guarantees the exact count of all induced subgraphs of size at most 4 using
just WL. The same idea can be extended further for larger patterns using
. We also compare the expressive power of Local WL with other GNN
hierarchies and show that given a bound on the time-complexity, our methods are
more expressive than the ones mentioned in Papp and Wattenhofer[2022a]
LIPIcs, Volume 261, ICALP 2023, Complete Volume
LIPIcs, Volume 261, ICALP 2023, Complete Volum
Neural function approximation on graphs: shape modelling, graph discrimination & compression
Graphs serve as a versatile mathematical abstraction of real-world phenomena in numerous scientific disciplines. This thesis is part of the Geometric Deep Learning subject area, a family of learning paradigms, that capitalise on the increasing volume of non-Euclidean data so as to solve real-world tasks in a data-driven manner. In particular, we focus on the topic of graph function approximation using neural networks, which lies at the heart of many relevant methods. In the first part of the thesis, we contribute to the understanding and design of Graph Neural Networks (GNNs). Initially, we investigate the problem of learning on signals supported on a fixed graph. We show that treating graph signals as general graph spaces is restrictive and conventional GNNs have limited expressivity. Instead, we expose a more enlightening perspective by drawing parallels between graph signals and signals on Euclidean grids, such as images and audio. Accordingly, we propose a permutation-sensitive GNN based on an operator analogous to shifts in grids and instantiate it on 3D meshes for shape modelling (Spiral Convolutions). Following, we focus on learning on general graph spaces and in particular on functions that are invariant to graph isomorphism. We identify a fundamental trade-off between invariance, expressivity and computational complexity, which we address with a symmetry-breaking mechanism based on substructure encodings (Graph Substructure Networks). Substructures are shown to be a powerful tool that provably improves expressivity while controlling computational complexity, and a useful inductive bias in network science and chemistry. In the second part of the thesis, we discuss the problem of graph compression, where we analyse the information-theoretic principles and the connections with graph generative models. We show that another inevitable trade-off surfaces, now between computational complexity and compression quality, due to graph isomorphism. We propose a substructure-based dictionary coder - Partition and Code (PnC) - with theoretical guarantees that can be adapted to different graph distributions by estimating its parameters from observations. Additionally, contrary to the majority of neural compressors, PnC is parameter and sample efficient and is therefore of wide practical relevance. Finally, within this framework, substructures are further illustrated as a decisive archetype for learning problems on graph spaces.Open Acces
Limitations of Game Comonads via Homomorphism Indistinguishability
Abramsky, Dawar, and Wang (2017) introduced the pebbling comonad for
k-variable counting logic and thereby initiated a line of work that imports
category theoretic machinery to finite model theory. Such game comonads have
been developed for various logics, yielding characterisations of logical
equivalences in terms of isomorphisms in the associated co-Kleisli category. We
show a first limitation of this approach by studying linear-algebraic logic,
which is strictly more expressive than first-order counting logic and whose
k-variable logical equivalence relations are known as invertible-map
equivalences (IM). We show that there exists no finite-rank comonad on the
category of graphs whose co-Kleisli isomorphisms characterise IM-equivalence,
answering a question of \'O Conghaile and Dawar (CSL 2021). We obtain this
result by ruling out a characterisation of IM-equivalence in terms of
homomorphism indistinguishability and employing the Lov\'asz-type theorems for
game comonads established by Dawar, Jakl, and Reggio (2021). Two graphs are
homomorphism indistinguishable over a graph class if they admit the same number
of homomorphisms from every graph in the class. The IM-equivalences cannot be
characterised in this way, neither when counting homomorphisms in the natural
numbers, nor in any finite prime field.Comment: Minor corrections in Section
Fundamentals
Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters
Fine-grained Expressivity of Graph Neural Networks
Numerous recent works have analyzed the expressive power of message-passing
graph neural networks (MPNNs), primarily utilizing combinatorial techniques
such as the -dimensional Weisfeiler-Leman test (-WL) for the graph
isomorphism problem. However, the graph isomorphism objective is inherently
binary, not giving insights into the degree of similarity between two given
graphs. This work resolves this issue by considering continuous extensions of
both -WL and MPNNs to graphons. Concretely, we show that the continuous
variant of -WL delivers an accurate topological characterization of the
expressive power of MPNNs on graphons, revealing which graphs these networks
can distinguish and the level of difficulty in separating them. We identify the
finest topology where MPNNs separate points and prove a universal approximation
theorem. Consequently, we provide a theoretical framework for graph and graphon
similarity combining various topological variants of classical
characterizations of the -WL. In particular, we characterize the expressive
power of MPNNs in terms of the tree distance, which is a graph distance based
on the concepts of fractional isomorphisms, and substructure counts via tree
homomorphisms, showing that these concepts have the same expressive power as
the -WL and MPNNs on graphons. Empirically, we validate our theoretical
findings by showing that randomly initialized MPNNs, without training, exhibit
competitive performance compared to their trained counterparts. Moreover, we
evaluate different MPNN architectures based on their ability to preserve graph
distances, highlighting the significance of our continuous -WL test in
understanding MPNNs' expressivity
The Complexity of Homomorphism Reconstructibility
Representing graphs by their homomorphism counts has led to the beautiful
theory of homomorphism indistinguishability in recent years. Moreover,
homomorphism counts have promising applications in database theory and machine
learning, where one would like to answer queries or classify graphs solely
based on the representation of a graph as a finite vector of homomorphism
counts from some fixed finite set of graphs to . We study the computational
complexity of the arguably most fundamental computational problem associated to
these representations, the homomorphism reconstructability problem: given a
finite sequence of graphs and a corresponding vector of natural numbers, decide
whether there exists a graph that realises the given vector as the
homomorphism counts from the given graphs.
We show that this problem yields a natural example of an
\mathsf{NP}^{#\mathsf{P}}-hard problem, which still can be -hard
when restricted to a fixed number of input graphs of bounded treewidth and a
fixed input vector of natural numbers, or alternatively, when restricted to a
finite input set of graphs. We further show that, when restricted to a finite
input set of graphs and given an upper bound on the order of the graph as
additional input, the problem cannot be -hard unless . For this regime, we obtain partial positive results. We also
investigate the problem's parameterised complexity and provide fpt-algorithms
for the case that a single graph is given and that multiple graphs of the same
order with subgraph instead of homomorphism counts are given