Graph Convolutional Neural Networks with Negative Sampling

University of Technology Sydney. Faculty of Engineering and Information Technology.Convolutional neural networks (CNNs) can learn potential features from large amounts of Euclidean data, such as text, images, which produce a satisfactory performance on pattern recognition and data mining. Besides Euclidean data, graphs, as non-Euclidean data, are powerful structures for modeling molecules, social networks, citation networks, traffic networks, etc. However, to lend learning power from the Euclidean space to a graph is not trivial. Learning with graphs requires an effective representation of their data structure. Graph Convolutional Neural Networks (GCNs) have been generally accepted to be an effective tool for node representations learning. An interesting way to understand GCNs is to think of them as a message-passing mechanism where each node updates its representation by accepting information from its neighbors (also known as positive samples). However, beyond these neighboring nodes, graphs have a large, dark, all-but-forgotten world in which we find the non-neighboring nodes (negative samples). In this thesis, we consider that this great dark world holds a substantial amount of information that might be useful for representation learning. Most specifically, it can provide negative information about the node representations. The key is how to select the appropriate negative samples for each node and incorporate the negative information contained in these samples into the representation update. We first propose a generalized method for fusing negative samples to graph convolutional neural networks. We next illustrate that the process of selecting the appropriate negative samples is not trivial. We proposed a determinantal point process algorithm-based method for efficiently obtaining samples. Finally, we use diverse negative samples to boost GCNs which jointly consider the positive and negative information when passing messages. The findings of this study not only have the development of the theory about negative samples in GCNs but also provide new ideas to alleviate the over-smoothing problem