10,301 research outputs found
Attention-based Graph Neural Network for Semi-supervised Learning
Recently popularized graph neural networks achieve the state-of-the-art
accuracy on a number of standard benchmark datasets for graph-based
semi-supervised learning, improving significantly over existing approaches.
These architectures alternate between a propagation layer that aggregates the
hidden states of the local neighborhood and a fully-connected layer. Perhaps
surprisingly, we show that a linear model, that removes all the intermediate
fully-connected layers, is still able to achieve a performance comparable to
the state-of-the-art models. This significantly reduces the number of
parameters, which is critical for semi-supervised learning where number of
labeled examples are small. This in turn allows a room for designing more
innovative propagation layers. Based on this insight, we propose a novel graph
neural network that removes all the intermediate fully-connected layers, and
replaces the propagation layers with attention mechanisms that respect the
structure of the graph. The attention mechanism allows us to learn a dynamic
and adaptive local summary of the neighborhood to achieve more accurate
predictions. In a number of experiments on benchmark citation networks
datasets, we demonstrate that our approach outperforms competing methods. By
examining the attention weights among neighbors, we show that our model
provides some interesting insights on how neighbors influence each other
Kernel-Induced Label Propagation by Mapping for Semi-Supervised Classification
Kernel methods have been successfully applied to the areas of pattern
recognition and data mining. In this paper, we mainly discuss the issue of
propagating labels in kernel space. A Kernel-Induced Label Propagation
(Kernel-LP) framework by mapping is proposed for high-dimensional data
classification using the most informative patterns of data in kernel space. The
essence of Kernel-LP is to perform joint label propagation and adaptive weight
learning in a transformed kernel space. That is, our Kernel-LP changes the task
of label propagation from the commonly-used Euclidean space in most existing
work to kernel space. The motivation of our Kernel-LP to propagate labels and
learn the adaptive weights jointly by the assumption of an inner product space
of inputs, i.e., the original linearly inseparable inputs may be mapped to be
separable in kernel space. Kernel-LP is based on existing positive and negative
LP model, i.e., the effects of negative label information are integrated to
improve the label prediction power. Also, Kernel-LP performs adaptive weight
construction over the same kernel space, so it can avoid the tricky process of
choosing the optimal neighborhood size suffered in traditional criteria. Two
novel and efficient out-of-sample approaches for our Kernel-LP to involve new
test data are also presented, i.e., (1) direct kernel mapping and (2) kernel
mapping-induced label reconstruction, both of which purely depend on the kernel
matrix between training set and testing set. Owing to the kernel trick, our
algorithms will be applicable to handle the high-dimensional real data.
Extensive results on real datasets demonstrate the effectiveness of our
approach.Comment: Accepted by IEEE TB
Low-rank Label Propagation for Semi-supervised Learning with 100 Millions Samples
The success of semi-supervised learning crucially relies on the scalability
to a huge amount of unlabelled data that are needed to capture the underlying
manifold structure for better classification. Since computing the pairwise
similarity between the training data is prohibitively expensive in most kinds
of input data, currently, there is no general ready-to-use semi-supervised
learning method/tool available for learning with tens of millions or more data
points. In this paper, we adopted the idea of two low-rank label propagation
algorithms, GLNP (Global Linear Neighborhood Propagation) and Kernel Nystr\"om
Approximation, and implemented the parallelized version of the two algorithms
accelerated with Nesterov's accelerated projected gradient descent for Big-data
Label Propagation (BigLP).
The parallel algorithms are tested on five real datasets ranging from 7000 to
10,000,000 in size and a simulation dataset of 100,000,000 samples. In the
experiments, the implementation can scale up to datasets with 100,000,000
samples and hundreds of features and the algorithms also significantly improved
the prediction accuracy when only a very small percentage of the data is
labeled. The results demonstrate that the BigLP implementation is highly
scalable to big data and effective in utilizing the unlabeled data for
semi-supervised learning
Fusion Graph Convolutional Networks
Semi-supervised node classification in attributed graphs, i.e., graphs with
node features, involves learning to classify unlabeled nodes given a partially
labeled graph. Label predictions are made by jointly modeling the node and its'
neighborhood features. State-of-the-art models for node classification on such
attributed graphs use differentiable recursive functions that enable
aggregation and filtering of neighborhood information from multiple hops. In
this work, we analyze the representation capacity of these models to regulate
information from multiple hops independently. From our analysis, we conclude
that these models despite being powerful, have limited representation capacity
to capture multi-hop neighborhood information effectively. Further, we also
propose a mathematically motivated, yet simple extension to existing graph
convolutional networks (GCNs) which has improved representation capacity. We
extensively evaluate the proposed model, F-GCN on eight popular datasets from
different domains. F-GCN outperforms the state-of-the-art models for
semi-supervised learning on six datasets while being extremely competitive on
the other two
Pairwise Constraint Propagation on Multi-View Data
This paper presents a graph-based learning approach to pairwise constraint
propagation on multi-view data. Although pairwise constraint propagation has
been studied extensively, pairwise constraints are usually defined over pairs
of data points from a single view, i.e., only intra-view constraint propagation
is considered for multi-view tasks. In fact, very little attention has been
paid to inter-view constraint propagation, which is more challenging since
pairwise constraints are now defined over pairs of data points from different
views. In this paper, we propose to decompose the challenging inter-view
constraint propagation problem into semi-supervised learning subproblems so
that they can be efficiently solved based on graph-based label propagation. To
the best of our knowledge, this is the first attempt to give an efficient
solution to inter-view constraint propagation from a semi-supervised learning
viewpoint. Moreover, since graph-based label propagation has been adopted for
basic optimization, we develop two constrained graph construction methods for
interview constraint propagation, which only differ in how the intra-view
pairwise constraints are exploited. The experimental results in cross-view
retrieval have shown the promising performance of our inter-view constraint
propagation
Semi-Supervised Representation Learning based on Probabilistic Labeling
In this paper, we present a new algorithm for semi-supervised representation
learning. In this algorithm, we first find a vector representation for the
labels of the data points based on their local positions in the space. Then, we
map the data to lower-dimensional space using a linear transformation such that
the dependency between the transformed data and the assigned labels is
maximized. In fact, we try to find a mapping that is as discriminative as
possible. The approach will use Hilber-Schmidt Independence Criterion (HSIC) as
the dependence measure. We also present a kernelized version of the algorithm,
which allows non-linear transformations and provides more flexibility in
finding the appropriate mapping. Use of unlabeled data for learning new
representation is not always beneficial and there is no algorithm that can
deterministically guarantee the improvement of the performance by exploiting
unlabeled data. Therefore, we also propose a bound on the performance of the
algorithm, which can be used to determine the effectiveness of using the
unlabeled data in the algorithm. We demonstrate the ability of the algorithm in
finding the transformation using both toy examples and real-world datasets.Comment: 8 pages, 7 figure
Deep graph learning for semi-supervised classification
Graph learning (GL) can dynamically capture the distribution structure (graph
structure) of data based on graph convolutional networks (GCN), and the
learning quality of the graph structure directly influences GCN for
semi-supervised classification. Existing methods mostly combine the
computational layer and the related losses into GCN for exploring the global
graph(measuring graph structure from all data samples) or local graph
(measuring graph structure from local data samples). Global graph emphasises on
the whole structure description of the inter-class data, while local graph
trend to the neighborhood structure representation of intra-class data.
However, it is difficult to simultaneously balance these graphs of the learning
process for semi-supervised classification because of the interdependence of
these graphs. To simulate the interdependence, deep graph learning(DGL) is
proposed to find the better graph representation for semi-supervised
classification. DGL can not only learn the global structure by the previous
layer metric computation updating, but also mine the local structure by next
layer local weight reassignment. Furthermore, DGL can fuse the different
structures by dynamically encoding the interdependence of these structures, and
deeply mine the relationship of the different structures by the hierarchical
progressive learning for improving the performance of semi-supervised
classification. Experiments demonstrate the DGL outperforms state-of-the-art
methods on three benchmark datasets (Citeseer,Cora, and Pubmed) for citation
networks and two benchmark datasets (MNIST and Cifar10) for images
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
There has been a surge of recent interest in learning representations for
graph-structured data. Graph representation learning methods have generally
fallen into three main categories, based on the availability of labeled data.
The first, network embedding (such as shallow graph embedding or graph
auto-encoders), focuses on learning unsupervised representations of relational
structure. The second, graph regularized neural networks, leverages graphs to
augment neural network losses with a regularization objective for
semi-supervised learning. The third, graph neural networks, aims to learn
differentiable functions over discrete topologies with arbitrary structure.
However, despite the popularity of these areas there has been surprisingly
little work on unifying the three paradigms. Here, we aim to bridge the gap
between graph neural networks, network embedding and graph regularization
models. We propose a comprehensive taxonomy of representation learning methods
for graph-structured data, aiming to unify several disparate bodies of work.
Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which
generalizes popular algorithms for semi-supervised learning on graphs (e.g.
GraphSage, Graph Convolutional Networks, Graph Attention Networks), and
unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc)
into a single consistent approach. To illustrate the generality of this
approach, we fit over thirty existing methods into this framework. We believe
that this unifying view both provides a solid foundation for understanding the
intuition behind these methods, and enables future research in the area
Semi-Supervised Learning on Graphs Based on Local Label Distributions
Most approaches that tackle the problem of node classification consider nodes
to be similar, if they have shared neighbors or are close to each other in the
graph. Recent methods for attributed graphs additionally take attributes of
neighboring nodes into account. We argue that the class labels of the neighbors
bear important information and considering them helps to improve classification
quality. Two nodes which are similar based on class labels in their
neighborhood do not need to be close-by in the graph and may even belong to
different connected components. In this work, we propose a novel approach for
the semi-supervised node classification. Precisely, we propose a new node
embedding which is based on the class labels in the local neighborhood of a
node. We show that this is a different setting from attribute-based embeddings
and thus, we propose a new method to learn label-based node embeddings which
can mirror a variety of relations between the class labels of neighboring
nodes. Our experimental evaluation demonstrates that our new methods can
significantly improve the prediction quality on real world data sets
Inductive Representation Learning on Large Graphs
Low-dimensional embeddings of nodes in large graphs have proved extremely
useful in a variety of prediction tasks, from content recommendation to
identifying protein functions. However, most existing approaches require that
all nodes in the graph are present during training of the embeddings; these
previous approaches are inherently transductive and do not naturally generalize
to unseen nodes. Here we present GraphSAGE, a general, inductive framework that
leverages node feature information (e.g., text attributes) to efficiently
generate node embeddings for previously unseen data. Instead of training
individual embeddings for each node, we learn a function that generates
embeddings by sampling and aggregating features from a node's local
neighborhood. Our algorithm outperforms strong baselines on three inductive
node-classification benchmarks: we classify the category of unseen nodes in
evolving information graphs based on citation and Reddit post data, and we show
that our algorithm generalizes to completely unseen graphs using a multi-graph
dataset of protein-protein interactions.Comment: Published in NIPS 2017; version with full appendix and minor
correction
- …