3 research outputs found
Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts
Predicting compound-protein affinity is critical for accelerating drug
discovery. Recent progress made by machine learning focuses on accuracy but
leaves much to be desired for interpretability. Through molecular contacts
underlying affinities, our large-scale interpretability assessment finds
commonly-used attention mechanisms inadequate. We thus formulate a hierarchical
multi-objective learning problem whose predicted contacts form the basis for
predicted affinities. We further design a physics-inspired deep relational
network, DeepRelations, with intrinsically explainable architecture.
Specifically, various atomic-level contacts or "relations" lead to
molecular-level affinity prediction. And the embedded attentions are
regularized with predicted structural contexts and supervised with partially
available training contacts. DeepRelations shows superior interpretability to
the state-of-the-art: without compromising affinity prediction, it boosts the
AUPRC of contact prediction 9.5, 16.9, 19.3 and 5.7-fold for the test,
compound-unique, protein-unique, and both-unique sets, respectively. Our study
represents the first dedicated model development and systematic model
assessment for interpretable machine learning of compound-protein affinity
When Does Self-Supervision Help Graph Convolutional Networks?
Self-supervision as an emerging technique has been employed to train
convolutional neural networks (CNNs) for more transferrable, generalizable, and
robust representation learning of images. Its introduction to graph
convolutional networks (GCNs) operating on graph data is however rarely
explored. In this study, we report the first systematic exploration and
assessment of incorporating self-supervision into GCNs. We first elaborate
three mechanisms to incorporate self-supervision into GCNs, analyze the
limitations of pretraining & finetuning and self-training, and proceed to focus
on multi-task learning. Moreover, we propose to investigate three novel
self-supervised learning tasks for GCNs with theoretical rationales and
numerical comparisons. Lastly, we further integrate multi-task self-supervision
into graph adversarial training. Our results show that, with properly designed
task forms and incorporation mechanisms, self-supervision benefits GCNs in
gaining more generalizability and robustness. Our codes are available at
https://github.com/Shen-Lab/SS-GCNs.Comment: Supplementary materials are available at
https://yyou1996.github.io/files/icml2020_ssgcn_supplement.pdf. ICML 202
A Unified Lottery Ticket Hypothesis for Graph Neural Networks
With graphs rapidly growing in size and deeper graph neural networks (GNNs)
emerging, the training and inference of GNNs become increasingly expensive.
Existing network weight pruning algorithms cannot address the main space and
computational bottleneck in GNNs, caused by the size and connectivity of the
graph. To this end, this paper first presents a unified GNN sparsification
(UGS) framework that simultaneously prunes the graph adjacency matrix and the
model weights, for effectively accelerating GNN inference on large-scale
graphs. Leveraging this new tool, we further generalize the recently popular
lottery ticket hypothesis to GNNs for the first time, by defining a graph
lottery ticket (GLT) as a pair of core sub-dataset and sparse sub-network,
which can be jointly identified from the original GNN and the full dense graph
by iteratively applying UGS. Like its counterpart in convolutional neural
networks, GLT can be trained in isolation to match the performance of training
with the full model and graph, and can be drawn from both randomly initialized
and self-supervised pre-trained GNNs. Our proposal has been experimentally
verified across various GNN architectures and diverse tasks, on both
small-scale graph datasets (Cora, Citeseer and PubMed), and large-scale
datasets from the challenging Open Graph Benchmark (OGB). Specifically, for
node classification, our found GLTs achieve the same accuracies with 20%~98%
MACs saving on small graphs and 25%~85% MACs saving on large ones. For link
prediction, GLTs lead to 48%~97% and 70% MACs saving on small and large graph
datasets, respectively, without compromising predictive performance. Codes
available at https://github.com/VITA-Group/Unified-LTH-GNN