30,248 research outputs found
Improving Bayesian Graph Convolutional Networks using Markov Chain Monte Carlo Graph Sampling
In the modern age of social media and networks, graph representations of real-world phenomena have become incredibly crucial. Often, we are interested in understanding how entities in a graph are interconnected. Graph Neural Networks (GNNs) have proven to be a very useful tool in a variety of graph learning tasks including node classification, link prediction, and edge classification. However, in most of these tasks, the graph data we are working with may be noisy and may contain spurious edges. That is, there is a lot of uncertainty associated with the underlying graph structure. Recent approaches to modeling uncertainty have been to use a Bayesian framework and view the graph as a random variable with probabilities associated with model parameters. Introducing the Bayesian paradigm to graph-based models, specifically for semi-supervised node classification, has been shown to yield higher classification accuracies. However, the method of graph inference proposed in recent work does not take into account the structure of the graph. In this paper, we propose Neighborhood Random Walk Sampling (NRWS), a Markov Chain Monte Carlo (MCMC) based graph sampling algorithm that utilizes graph structure, improves diversity among connections, and yields consistently competitive classification results compared to the state-of-the-art in semi-supervised node classification
Deep collective inference
Collective inference is widely used to improve classification in network datasets. However, despite recent advances in deep learning and the successes of recurrent neural networks (RNNs), researchers have only just recently begun to study how to apply RNNs to heterogeneous graph and network datasets. There has been recent work on using RNNs for unsupervised learning in networks (e.g., graph clustering, node embedding) and for prediction (e.g., link prediction, graph classification), but there has been little work on using RNNs for node-based relational classification tasks. In this paper, we provide an end-to-end learning framework using RNNs for collective inference. Our main insight is to transform a node and its set of neighbors into an unordered sequence (of varying length) and use an LSTM-based RNN to predict the class label as the output of that sequence. We develop a collective inference method, which we refer to as Deep Collective Inference (DCI), that uses semi-supervised learning in partially-labeled networks and two label distribution correction mechanisms for imbalanced classes. We compare to several alternative methods on seven network datasets. DCI achieves up to a 12% reduction in error compared to the best alternative and a 25% reduction in error on average over all methods, for all label proportions
Adapting to Change: Robust Counterfactual Explanations in Dynamic Data Landscapes
We introduce a novel semi-supervised Graph Counterfactual Explainer (GCE)
methodology, Dynamic GRAph Counterfactual Explainer (DyGRACE). It leverages
initial knowledge about the data distribution to search for valid
counterfactuals while avoiding using information from potentially outdated
decision functions in subsequent time steps. Employing two graph autoencoders
(GAEs), DyGRACE learns the representation of each class in a binary
classification scenario. The GAEs minimise the reconstruction error between the
original graph and its learned representation during training. The method
involves (i) optimising a parametric density function (implemented as a
logistic regression function) to identify counterfactuals by maximising the
factual autoencoder's reconstruction error, (ii) minimising the counterfactual
autoencoder's error, and (iii) maximising the similarity between the factual
and counterfactual graphs. This semi-supervised approach is independent of an
underlying black-box oracle. A logistic regression model is trained on a set of
graph pairs to learn weights that aid in finding counterfactuals. At inference,
for each unseen graph, the logistic regressor identifies the best
counterfactual candidate using these learned weights, while the GAEs can be
iteratively updated to represent the continual adaptation of the learned graph
representation over iterations. DyGRACE is quite effective and can act as a
drift detector, identifying distributional drift based on differences in
reconstruction errors between iterations. It avoids reliance on the oracle's
predictions in successive iterations, thereby increasing the efficiency of
counterfactual discovery. DyGRACE, with its capacity for contrastive learning
and drift detection, will offer new avenues for semi-supervised learning and
explanation generation
Addressing the Impact of Localized Training Data in Graph Neural Networks
Graph Neural Networks (GNNs) have achieved notable success in learning from
graph-structured data, owing to their ability to capture intricate dependencies
and relationships between nodes. They excel in various applications, including
semi-supervised node classification, link prediction, and graph generation.
However, it is important to acknowledge that the majority of state-of-the-art
GNN models are built upon the assumption of an in-distribution setting, which
hinders their performance on real-world graphs with dynamic structures. In this
article, we aim to assess the impact of training GNNs on localized subsets of
the graph. Such restricted training data may lead to a model that performs well
in the specific region it was trained on but fails to generalize and make
accurate predictions for the entire graph. In the context of graph-based
semi-supervised learning (SSL), resource constraints often lead to scenarios
where the dataset is large, but only a portion of it can be labeled, affecting
the model's performance. This limitation affects tasks like anomaly detection
or spam detection when labeling processes are biased or influenced by human
subjectivity. To tackle the challenges posed by localized training data, we
approach the problem as an out-of-distribution (OOD) data issue by by aligning
the distributions between the training data, which represents a small portion
of labeled data, and the graph inference process that involves making
predictions for the entire graph. We propose a regularization method to
minimize distributional discrepancies between localized training data and graph
inference, improving model performance on OOD data. Extensive tests on popular
GNN models show significant performance improvement on three citation GNN
benchmark datasets. The regularization approach effectively enhances model
adaptation and generalization, overcoming challenges posed by OOD data.Comment: 6 pages, 4 figure
Bayesian Semi-supervised Learning with Graph Gaussian Processes
We propose a data-efficient Gaussian process-based Bayesian approach to the
semi-supervised learning problem on graphs. The proposed model shows extremely
competitive performance when compared to the state-of-the-art graph neural
networks on semi-supervised learning benchmark experiments, and outperforms the
neural networks in active learning experiments where labels are scarce.
Furthermore, the model does not require a validation data set for early
stopping to control over-fitting. Our model can be viewed as an instance of
empirical distribution regression weighted locally by network connectivity. We
further motivate the intuitive construction of the model with a Bayesian linear
model interpretation where the node features are filtered by an operator
related to the graph Laplacian. The method can be easily implemented by
adapting off-the-shelf scalable variational inference algorithms for Gaussian
processes.Comment: To appear in NIPS 2018 Fixed an error in Figure 2. The previous arxiv
version contains two identical sub-figure
- …