On Filter Size in Graph Convolutional Networks

Recently, many researchers have been focusing on the definition of neural networks for graphs. The basic component for many of these approaches remains the graph convolution idea proposed almost a decade ago. In this paper, we extend this basic component, following an intuition derived from the well-known convolutional filters over multi-dimensional tensors. In particular, we derive a simple, efficient and effective way to introduce a hyper-parameter on graph convolutions that influences the filter size, i.e. its receptive field over the considered graph. We show with experimental results on real-world graph datasets that the proposed graph convolutional filter improves the predictive performance of Deep Graph Convolutional Networks.Comment: arXiv admin note: text overlap with arXiv:1811.0693

Fast and Deep Graph Neural Networks

We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification.Comment: Pre-print of 'Fast and Deep Graph Neural Networks', accepted for AAAI 2020. This document includes the Supplementary Materia

Shapes of uncertainty in spectral graph theory

We present a flexible framework for uncertainty principles in spectral graph theory. In this framework, general filter functions modeling the spatial and spectral localization of a graph signal can be incorporated. It merges several existing uncertainty relations on graphs, among others the Landau-Pollak principle describing the joint admissibility region of two projection operators, and uncertainty relations based on spectral and spatial spreads. Using theoretical and computational aspects of the numerical range of matrices, we are able to characterize and illustrate the shapes of the uncertainty curves and to study the space-frequency localization of signals inside the admissibility regions

Computing graph neural networks: A survey from algorithms to accelerators

Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data are inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of ground-breaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this article aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.This work is possible thanks to funding from the European Union’s Horizon 2020 research and innovation programme under Grant No. 863337 (WiPLASH project) and the Spanish Ministry of Economy and Competitiveness under contract TEC2017-90034-C2-1-R (ALLIANCE project) that receives funding from FEDER.Peer ReviewedPostprint (published version

Deep learning structure for directed graph data

Deep learning structures have achieved outstanding success in many different domains. Existing research works have proposed and presented many state-of-the-art deep neural networks to solve different learning tasks in various research fields such as speech processing and image recognition. Graph neural networks (GNNs) are considered as a type of deep neural network and their numerical representation from the graph does improve the performance of networks. In the real-world cases, data is not only in the form of simple graph, but also they could contain direction information in the graph resulting in the so-called directed graph data. This thesis will introduce and explain the first attempt in this domain to apply Singular Value Decomposition (SVD) on adjacency matrix for graph convolutional neural networks and propose SVD-GCN. This thesis also utilizes the framelet decomposition to help better filter the graph signals, thus to improve novel structure’s performance in node classification task and to enhance the robustness of the model when encountering high-level noise attack. The thesis also applies the new model on link prediction tasks. All the experimental results demonstrate SVD-GCN’s outstanding performances in both node-level and edgelevel learning tasks