90 research outputs found
HiHGNN: Accelerating HGNNs through Parallelism and Data Reusability Exploitation
Heterogeneous graph neural networks (HGNNs) have emerged as powerful
algorithms for processing heterogeneous graphs (HetGs), widely used in many
critical fields. To capture both structural and semantic information in HetGs,
HGNNs first aggregate the neighboring feature vectors for each vertex in each
semantic graph and then fuse the aggregated results across all semantic graphs
for each vertex. Unfortunately, existing graph neural network accelerators are
ill-suited to accelerate HGNNs. This is because they fail to efficiently tackle
the specific execution patterns and exploit the high-degree parallelism as well
as data reusability inside and across the processing of semantic graphs in
HGNNs.
In this work, we first quantitatively characterize a set of representative
HGNN models on GPU to disclose the execution bound of each stage,
inter-semantic-graph parallelism, and inter-semantic-graph data reusability in
HGNNs. Guided by our findings, we propose a high-performance HGNN accelerator,
HiHGNN, to alleviate the execution bound and exploit the newfound parallelism
and data reusability in HGNNs. Specifically, we first propose a bound-aware
stage-fusion methodology that tailors to HGNN acceleration, to fuse and
pipeline the execution stages being aware of their execution bounds. Second, we
design an independency-aware parallel execution design to exploit the
inter-semantic-graph parallelism. Finally, we present a similarity-aware
execution scheduling to exploit the inter-semantic-graph data reusability.
Compared to the state-of-the-art software framework running on NVIDIA GPU T4
and GPU A100, HiHGNN respectively achieves an average 41.5 and
8.6 speedup as well as 106 and 73 energy efficiency
with quarter the memory bandwidth of GPU A100
Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design
Graph neural networks (GNNs) have shown significant accuracy improvements in
a variety of graph learning domains, sparking considerable research interest.
To translate these accuracy improvements into practical applications, it is
essential to develop high-performance and efficient hardware acceleration for
GNN models. However, designing GNN accelerators faces two fundamental
challenges: the high bandwidth requirement of GNN models and the diversity of
GNN models. Previous works have addressed the first challenge by using more
expensive memory interfaces to achieve higher bandwidth. For the second
challenge, existing works either support specific GNN models or have generic
designs with poor hardware utilization.
In this work, we tackle both challenges simultaneously. First, we identify a
new type of partition-level operator fusion, which we utilize to internally
reduce the high bandwidth requirement of GNNs. Next, we introduce
partition-level multi-threading to schedule the concurrent processing of graph
partitions, utilizing different hardware resources. To further reduce the extra
on-chip memory required by multi-threading, we propose fine-grained graph
partitioning to generate denser graph partitions. Importantly, these three
methods make no assumptions about the targeted GNN models, addressing the
challenge of model variety. We implement these methods in a framework called
SwitchBlade, consisting of a compiler, a graph partitioner, and a hardware
accelerator. Our evaluation demonstrates that SwitchBlade achieves an average
speedup of and energy savings of compared to the
NVIDIA V100 GPU. Additionally, SwitchBlade delivers performance comparable to
state-of-the-art specialized accelerators
Characterizing the Influence of Graph Elements
Influence function, a method from robust statistics, measures the changes of
model parameters or some functions about model parameters concerning the
removal or modification of training instances. It is an efficient and useful
post-hoc method for studying the interpretability of machine learning models
without the need for expensive model re-training. Recently, graph convolution
networks (GCNs), which operate on graph data, have attracted a great deal of
attention. However, there is no preceding research on the influence functions
of GCNs to shed light on the effects of removing training nodes/edges from an
input graph. Since the nodes/edges in a graph are interdependent in GCNs, it is
challenging to derive influence functions for GCNs. To fill this gap, we
started with the simple graph convolution (SGC) model that operates on an
attributed graph and formulated an influence function to approximate the
changes in model parameters when a node or an edge is removed from an
attributed graph. Moreover, we theoretically analyzed the error bound of the
estimated influence of removing an edge. We experimentally validated the
accuracy and effectiveness of our influence estimation function. In addition,
we showed that the influence function of an SGC model could be used to estimate
the impact of removing training nodes/edges on the test performance of the SGC
without re-training the model. Finally, we demonstrated how to use influence
functions to guide the adversarial attacks on GCNs effectively
Computing graph neural networks: A survey from algorithms to accelerators
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data are inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of ground-breaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this article aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.This work is possible thanks to funding from the European Union’s Horizon 2020 research and innovation programme under Grant No. 863337 (WiPLASH project) and the Spanish Ministry of Economy and Competitiveness under contract TEC2017-90034-C2-1-R (ALLIANCE project) that receives funding from FEDER.Peer ReviewedPostprint (published version
Hierarchical Graph Convolutional Network Built by Multiscale Atlases for Brain Disorder Diagnosis Using Functional Connectivity
Functional connectivity network (FCN) data from functional magnetic resonance
imaging (fMRI) is increasingly used for the diagnoses of brain disorders.
However, state-of-the-art studies used to build the FCN using a single brain
parcellation atlas at a certain spatial scale, which largely neglected
functional interactions across different spatial scales in hierarchical
manners. In this study, we propose a novel framework to perform multiscale FCN
analysis for brain disorder diagnosis. We first use a set of well-defined
multiscale atlases to compute multiscale FCNs. Then, we utilize biologically
meaningful brain hierarchical relationships among the regions in multiscale
atlases to perform nodal pooling across multiple spatial scales, namely
"Atlas-guided Pooling". Accordingly, we propose a Multiscale-Atlases-based
Hierarchical Graph Convolutional Network (MAHGCN), built on the stacked layers
of graph convolution and the atlas-guided pooling, for a comprehensive
extraction of diagnostic information from multiscale FCNs. Experiments on
neuroimaging data from 1792 subjects demonstrate the effectiveness of our
proposed method in the diagnoses of Alzheimer's disease (AD), the prodromal
stage of AD (i.e., mild cognitive impairment [MCI]), as well as autism spectrum
disorder (ASD), with accuracy of 88.9%, 78.6%, and 72.7% respectively. All
results show significant advantages of our proposed method over other competing
methods. This study not only demonstrates the feasibility of brain disorder
diagnosis using resting-state fMRI empowered by deep learning, but also
highlights that the functional interactions in the multiscale brain hierarchy
are worth being explored and integrated into deep learning network
architectures for better understanding the neuropathology of brain disorders
- …