1,386 research outputs found
CardiGraphormer: Unveiling the Power of Self-Supervised Learning in Revolutionizing Drug Discovery
In the expansive realm of drug discovery, with approximately 15,000 known
drugs and only around 4,200 approved, the combinatorial nature of the chemical
space presents a formidable challenge. While Artificial Intelligence (AI) has
emerged as a powerful ally, traditional AI frameworks face significant hurdles.
This manuscript introduces CardiGraphormer, a groundbreaking approach that
synergizes self-supervised learning (SSL), Graph Neural Networks (GNNs), and
Cardinality Preserving Attention to revolutionize drug discovery.
CardiGraphormer, a novel combination of Graphormer and Cardinality Preserving
Attention, leverages SSL to learn potent molecular representations and employs
GNNs to extract molecular fingerprints, enhancing predictive performance and
interpretability while reducing computation time. It excels in handling complex
data like molecular structures and performs tasks associated with nodes, pairs
of nodes, subgraphs, or entire graph structures. CardiGraphormer's potential
applications in drug discovery and drug interactions are vast, from identifying
new drug targets to predicting drug-to-drug interactions and enabling novel
drug discovery. This innovative approach provides an AI-enhanced methodology in
drug development, utilizing SSL combined with GNNs to overcome existing
limitations and pave the way for a richer exploration of the vast combinatorial
chemical space in drug discovery
Optimization and Application of Graph Neural Networks
Graph Neural Networks (GNNs) are widely recognized for their potential in learning from graph-structured data and solving complex problems. However, optimal performance and applicability of GNNs have been an open-ended challenge. This dissertation presents a series of substantial advances addressing this problem. First, we investigate attention-based GNNs, revealing a critical shortcoming: their ignorance of cardinality information that impacts their discriminative power. To rectify this, we propose Cardinality Preserved Attention (CPA) models that can be applied to any attention-based GNNs, which exhibit a marked improvement in performance. Next, we introduce the Directional Node Pair (DNP) descriptor and the Robust Molecular Graph Convolutional Network (RoM-GCN). These are robust tools designed to encode both chemical and geometric information, ensuring an accurate representation of 3D molecules. Finally, we develop Physics-Aware Multiplex Graph Neural Network (PAMNet), a universal, physics-informed framework that models 3D molecular systems with high accuracy and efficiency. This innovation is particularly effective in a variety of molecular tasks, outperforming existing baselines. Collectively, these advances underscore the need for continued exploration of bespoke optimization strategies to fully realize the potential of GNNs across different application domains
A Survey on Graph Representation Learning Methods
Graphs representation learning has been a very active research area in recent
years. The goal of graph representation learning is to generate graph
representation vectors that capture the structure and features of large graphs
accurately. This is especially important because the quality of the graph
representation vectors will affect the performance of these vectors in
downstream tasks such as node classification, link prediction and anomaly
detection. Many techniques are proposed for generating effective graph
representation vectors. Two of the most prevalent categories of graph
representation learning are graph embedding methods without using graph neural
nets (GNN), which we denote as non-GNN based graph embedding methods, and graph
neural nets (GNN) based methods. Non-GNN graph embedding methods are based on
techniques such as random walks, temporal point processes and neural network
learning methods. GNN-based methods, on the other hand, are the application of
deep learning on graph data. In this survey, we provide an overview of these
two categories and cover the current state-of-the-art methods for both static
and dynamic graphs. Finally, we explore some open and ongoing research
directions for future work
Serving Graph Neural Networks With Distributed Fog Servers For Smart IoT Services
Graph Neural Networks (GNNs) have gained growing interest in miscellaneous
applications owing to their outstanding ability in extracting latent
representation on graph structures. To render GNN-based service for IoT-driven
smart applications, traditional model serving paradigms usually resort to the
cloud by fully uploading geo-distributed input data to remote datacenters.
However, our empirical measurements reveal the significant communication
overhead of such cloud-based serving and highlight the profound potential in
applying the emerging fog computing. To maximize the architectural benefits
brought by fog computing, in this paper, we present Fograph, a novel
distributed real-time GNN inference framework that leverages diverse and
dynamic resources of multiple fog nodes in proximity to IoT data sources. By
introducing heterogeneity-aware execution planning and GNN-specific compression
techniques, Fograph tailors its design to well accommodate the unique
characteristics of GNN serving in fog environments. Prototype-based evaluation
and case study demonstrate that Fograph significantly outperforms the
state-of-the-art cloud serving and fog deployment by up to 5.39x execution
speedup and 6.84x throughput improvement.Comment: Accepted by IEEE/ACM Transactions on Networkin
struc2gauss: Structural role preserving network embedding via Gaussian embedding
Network embedding (NE) is playing a principal role in network mining, due to its ability to map nodes into efficient low-dimensional embedding vectors. However, two major limitations exist in state-of-the-art NE methods: role preservation and uncertainty modeling. Almost all previous methods represent a node into a point in space and focus on local structural information, i.e., neighborhood information. However, neighborhood information does not capture global structural information and point vector representation fails in modeling the uncertainty of node representations. In this paper, we propose a new NE framework, struc2gauss, which learns node representations in the space of Gaussian distributions and performs network embedding based on global structural information. struc2gauss first employs a given node similarity metric to measure the global structural information, then generates structural context for nodes and finally learns node representations via Gaussian embedding. Different structural similarity measures of networks and energy functions of Gaussian embedding are investigated. Experiments conducted on real-world networks demonstrate that struc2gauss effectively captures global structural information while state-of-the-art network embedding methods fail to, outperforms other methods on the structure-based clustering and classification task and provides more information on uncertainties of node representations
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
Sketch as an image search query is an ideal alternative to text in capturing
the fine-grained visual details. Prior successes on fine-grained sketch-based
image retrieval (FG-SBIR) have demonstrated the importance of tackling the
unique traits of sketches as opposed to photos, e.g., temporal vs. static,
strokes vs. pixels, and abstract vs. pixel-perfect. In this paper, we study a
further trait of sketches that has been overlooked to date, that is, they are
hierarchical in terms of the levels of detail -- a person typically sketches up
to various extents of detail to depict an object. This hierarchical structure
is often visually distinct. In this paper, we design a novel network that is
capable of cultivating sketch-specific hierarchies and exploiting them to match
sketch with photo at corresponding hierarchical levels. In particular, features
from a sketch and a photo are enriched using cross-modal co-attention, coupled
with hierarchical node fusion at every level to form a better embedding space
to conduct retrieval. Experiments on common benchmarks show our method to
outperform state-of-the-arts by a significant margin.Comment: Accepted for ORAL presentation in BMVC 202
Heterogeneous Multi-Layered Network Model for Omics Data Integration and Analysis
Advances in next-generation sequencing and high-throughput techniques have enabled the generation of vast amounts of diverse omics data. These big data provide an unprecedented opportunity in biology, but impose great challenges in data integration, data mining, and knowledge discovery due to the complexity, heterogeneity, dynamics, uncertainty, and high-dimensionality inherited in the omics data. Network has been widely used to represent relations between entities in biological system, such as protein-protein interaction, gene regulation, and brain connectivity (i.e. network construction) as well as to infer novel relations given a reconstructed network (aka link prediction). Particularly, heterogeneous multi-layered network (HMLN) has proven successful in integrating diverse biological data for the representation of the hierarchy of biological system. The HMLN provides unparalleled opportunities but imposes new computational challenges on establishing causal genotype-phenotype associations and understanding environmental impact on organisms. In this review, we focus on the recent advances in developing novel computational methods for the inference of novel biological relations from the HMLN. We first discuss the properties of biological HMLN. Then we survey four categories of state-of-the-art methods (matrix factorization, random walk, knowledge graph, and deep learning). Thirdly, we demonstrate their applications to omics data integration and analysis. Finally, we outline strategies for future directions in the development of new HMLN models
Not All Neighbors Are Worth Attending to: Graph Selective Attention Networks for Semi-supervised Learning
Graph attention networks (GATs) are powerful tools for analyzing graph data
from various real-world scenarios. To learn representations for downstream
tasks, GATs generally attend to all neighbors of the central node when
aggregating the features. In this paper, we show that a large portion of the
neighbors are irrelevant to the central nodes in many real-world graphs, and
can be excluded from neighbor aggregation. Taking the cue, we present Selective
Attention (SA) and a series of novel attention mechanisms for graph neural
networks (GNNs). SA leverages diverse forms of learnable node-node
dissimilarity to acquire the scope of attention for each node, from which
irrelevant neighbors are excluded. We further propose Graph selective attention
networks (SATs) to learn representations from the highly correlated node
features identified and investigated by different SA mechanisms. Lastly,
theoretical analysis on the expressive power of the proposed SATs and a
comprehensive empirical study of the SATs on challenging real-world datasets
against state-of-the-art GNNs are presented to demonstrate the effectiveness of
SATs
- …