3 research outputs found
Molecule Property Prediction and Classification with Graph Hypernetworks
Graph neural networks are currently leading the performance charts in
learning-based molecule property prediction and classification. Computational
chemistry has, therefore, become the a prominent testbed for generic graph
neural networks, as well as for specialized message passing methods. In this
work, we demonstrate that the replacement of the underlying networks with
hypernetworks leads to a boost in performance, obtaining state of the art
results in various benchmarks. A major difficulty in the application of
hypernetworks is their lack of stability. We tackle this by combining the
current message and the first message. A recent work has tackled the training
instability of hypernetworks in the context of error correcting codes, by
replacing the activation function of the message passing network with a
low-order Taylor approximation of it. We demonstrate that our generic solution
can replace this domain-specific solution
A Gated Hypernet Decoder for Polar Codes
Hypernetworks were recently shown to improve the performance of message
passing algorithms for decoding error correcting codes. In this work, we
demonstrate how hypernetworks can be applied to decode polar codes by employing
a new formalization of the polar belief propagation decoding scheme. We
demonstrate that our method improves the previous results of neural polar
decoders and achieves, for large SNRs, the same bit-error-rate performances as
the successive list cancellation method, which is known to be better than any
belief propagation decoders and very close to the maximum likelihood decoder.Comment: Accepted to ICASSP 202
From Local Structures to Size Generalization in Graph Neural Networks
Graph neural networks (GNNs) can process graphs of different sizes, but their
ability to generalize across sizes, specifically from small to large graphs, is
still not well understood. In this paper, we identify an important type of data
where generalization from small to large graphs is challenging: graph
distributions for which the local structure depends on the graph size. This
effect occurs in multiple important graph learning domains, including social
and biological networks. We first prove that when there is a difference between
the local structures, GNNs are not guaranteed to generalize across sizes: there
are "bad" global minima that do well on small graphs but fail on large graphs.
We then study the size-generalization problem empirically and demonstrate that
when there is a discrepancy in local structure, GNNs tend to converge to
non-generalizing solutions. Finally, we suggest two approaches for improving
size generalization, motivated by our findings. Notably, we propose a novel
Self-Supervised Learning (SSL) task aimed at learning meaningful
representations of local structures that appear in large graphs. Our SSL task
improves classification accuracy on several popular datasets.Comment: Camera ready version for ICML 202