53 research outputs found
Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View
Graph Neural Networks (GNNs) have achieved promising performance on a wide
range of graph-based tasks. Despite their success, one severe limitation of
GNNs is the over-smoothing issue (indistinguishable representations of nodes in
different classes). In this work, we present a systematic and quantitative
study on the over-smoothing issue of GNNs. First, we introduce two quantitative
metrics, MAD and MADGap, to measure the smoothness and over-smoothness of the
graph nodes representations, respectively. Then, we verify that smoothing is
the nature of GNNs and the critical factor leading to over-smoothness is the
low information-to-noise ratio of the message received by the nodes, which is
partially determined by the graph topology. Finally, we propose two methods to
alleviate the over-smoothing issue from the topological view: (1) MADReg which
adds a MADGap-based regularizer to the training objective;(2) AdaGraph which
optimizes the graph topology based on the model predictions. Extensive
experiments on 7 widely-used graph datasets with 10 typical GNN models show
that the two proposed methods are effective for relieving the over-smoothing
issue, thus improving the performance of various GNN models.Comment: Accepted by AAAI 2020. This complete version contains the appendi
Directed Message Passing Based on Attention for Prediction of Molecular Properties
Molecular representation learning (MRL) has long been crucial in the fields
of drug discovery and materials science, and it has made significant progress
due to the development of natural language processing (NLP) and graph neural
networks (GNNs). NLP treats the molecules as one dimensional sequential tokens
while GNNs treat them as two dimensional topology graphs. Based on different
message passing algorithms, GNNs have various performance on detecting chemical
environments and predicting molecular properties. Herein, we propose Directed
Graph Attention Networks (D-GATs): the expressive GNNs with directed bonds. The
key to the success of our strategy is to treat the molecular graph as directed
graph and update the bond states and atom states by scaled dot-product
attention mechanism. This allows the model to better capture the sub-structure
of molecular graph, i.e., functional groups. Compared to other GNNs or Message
Passing Neural Networks (MPNNs), D-GATs outperform the state-of-the-art on 13
out of 15 important molecular property prediction benchmarks.Comment: Computational Materials Science, In pres
Multiparameter Persistent Homology for Molecular Property Prediction
In this study, we present a novel molecular fingerprint generation method
based on multiparameter persistent homology. This approach reveals the latent
structures and relationships within molecular geometry, and detects topological
features that exhibit persistence across multiple scales along multiple
parameters, such as atomic mass, partial charge, and bond type, and can be
further enhanced by incorporating additional parameters like ionization energy,
electron affinity, chirality and orbital hybridization. The proposed
fingerprinting method provides fresh perspectives on molecular structure that
are not easily discernible from single-parameter or single-scale analysis.
Besides, in comparison with traditional graph neural networks, multiparameter
persistent homology has the advantage of providing a more comprehensive and
interpretable characterization of the topology of the molecular data. We have
established theoretical stability guarantees for multiparameter persistent
homology, and have conducted extensive experiments on the Lipophilicity,
FreeSolv, and ESOL datasets to demonstrate its effectiveness in predicting
molecular properties.Comment: ICLR 2023-Machine Learning for Drug Discovery. arXiv admin note: text
overlap with arXiv:2211.0380
ifMixup: Interpolating Graph Pair to Regularize Graph Classification
We present a simple and yet effective interpolation-based regularization
technique, aiming to improve the generalization of Graph Neural Networks (GNNs)
on supervised graph classification. We leverage Mixup, an effective regularizer
for vision, where random sample pairs and their labels are interpolated to
create synthetic images for training. Unlike images with grid-like coordinates,
graphs have arbitrary structure and topology, which can be very sensitive to
any modification that alters the graph's semantic meanings. This posts two
unanswered questions for Mixup-like regularization schemes: Can we directly mix
up a pair of graph inputs? If so, how well does such mixing strategy regularize
the learning of GNNs? To answer these two questions, we propose ifMixup, which
first adds dummy nodes to make two graphs have the same input size and then
simultaneously performs linear interpolation between the aligned node feature
vectors and the aligned edge representations of the two graphs. We empirically
show that such simple mixing schema can effectively regularize the
classification learning, resulting in superior predictive accuracy to popular
graph augmentation and GNN methods.Comment: To appear in AAAI202
- …