21,269 research outputs found
Deep Feature Learning of Multi-Network Topology for Node Classification
Networks are ubiquitous structure that describes complex relationships
between different entities in the real world. As a critical component of
prediction task over nodes in networks, learning the feature representation of
nodes has become one of the most active areas recently. Network Embedding,
aiming to learn non-linear and low-dimensional feature representation based on
network topology, has been proved to be helpful on tasks of network analysis,
especially node classification. For many real-world systems, multiple types of
relations are naturally represented by multiple networks. However, existing
network embedding methods mainly focus on single network embedding and neglect
the information shared among different networks. In this paper, we propose a
novel multiple network embedding method based on semisupervised autoencoder,
named DeepMNE, which captures complex topological structures of multi-networks
and takes the correlation among multi-networks into account. We evaluate
DeepMNE on the task of node classification with two real-world datasets. The
experimental results demonstrate the superior performance of our method over
four state-of-the-art algorithms
Expanding Label Sets for Graph Convolutional Networks
In recent years, Graph Convolutional Networks (GCNs) and their variants have
been widely utilized in learning tasks that involve graphs. These tasks include
recommendation systems, node classification, among many others. In node
classification problem, the input is a graph in which the edges represent the
association between pairs of nodes, multi-dimensional feature vectors are
associated with the nodes, and some of the nodes in the graph have known
labels. The objective is to predict the labels of the nodes that are not
labeled, using the nodes features, in conjunction with graph topology. While
GCNs have been successfully applied to this problem, the caveats that they
inherit from traditional deep learning models pose significant challenges to
broad utilization of GCNs in node classification. One such caveat is that
training a GCN requires a large number of labeled training instances, which is
often not the case in realistic settings. To remedy this requirement,
state-of-the-art methods leverage network diffusion-based approaches to
propagate labels across the network before training GCNs. However, these
approaches ignore the tendency of the network diffusion methods in biasing
proximity with centrality, resulting in the propagation of labels to the nodes
that are well-connected in the graph. To address this problem, here we present
an alternate approach to extrapolating node labels in GCNs in the following
three steps: (i) clustering of the network to identify communities, (ii) use of
network diffusion algorithms to quantify the proximity of each node to the
communities, thereby obtaining a low-dimensional topological profile for each
node, (iii) comparing these topological profiles to identify nodes that are
most similar to the labeled nodes
Graph Embedding with Rich Information through Heterogeneous Network
Graph embedding has attracted increasing attention due to its critical
application in social network analysis. Most existing algorithms for graph
embedding only rely on the typology information and fail to use the copious
information in nodes as well as edges. As a result, their performance for many
tasks may not be satisfactory. In this paper, we proposed a novel and general
framework of representation learning for graph with rich text information
through constructing a bipartite heterogeneous network. Specially, we designed
a biased random walk to explore the constructed heterogeneous network with the
notion of flexible neighborhood. The efficacy of our method is demonstrated by
extensive comparison experiments with several baselines on various datasets. It
improves the Micro-F1 and Macro-F1 of node classification by 10% and 7% on Cora
dataset.Comment: 9 pages, 7 figures, 4 table
Multi-Hot Compact Network Embedding
Network embedding, as a promising way of the network representation learning,
is capable of supporting various subsequent network mining and analysis tasks,
and has attracted growing research interests recently. Traditional approaches
assign each node with an independent continuous vector, which will cause huge
memory overhead for large networks. In this paper we propose a novel multi-hot
compact embedding strategy to effectively reduce memory cost by learning
partially shared embeddings. The insight is that a node embedding vector is
composed of several basis vectors, which can significantly reduce the number of
continuous vectors while maintain similar data representation ability.
Specifically, we propose a MCNE model to learn compact embeddings from
pre-learned node features. A novel component named compressor is integrated
into MCNE to tackle the challenge that popular back-propagation optimization
cannot propagate through discrete samples. We further propose an end-to-end
model MCNE to learn compact embeddings from the input network directly.
Empirically, we evaluate the proposed models over three real network datasets,
and the results demonstrate that our proposals can save about 90\% of memory
cost of network embeddings without significantly performance decline
A Review of Modularization Techniques in Artificial Neural Networks
Artificial neural networks (ANNs) have achieved significant success in
tackling classical and modern machine learning problems. As learning problems
grow in scale and complexity, and expand into multi-disciplinary territory, a
more modular approach for scaling ANNs will be needed. Modular neural networks
(MNNs) are neural networks that embody the concepts and principles of
modularity. MNNs adopt a large number of different techniques for achieving
modularization. Previous surveys of modularization techniques are relatively
scarce in their systematic analysis of MNNs, focusing mostly on empirical
comparisons and lacking an extensive taxonomical framework. In this review, we
aim to establish a solid taxonomy that captures the essential properties and
relationships of the different variants of MNNs. Based on an investigation of
the different levels at which modularization techniques act, we attempt to
provide a universal and systematic framework for theorists studying MNNs, also
trying along the way to emphasise the strengths and weaknesses of different
modularization approaches in order to highlight good practices for neural
network practitioners.Comment: Artif Intell Rev (2019
Classification of EEG-Based Brain Connectivity Networks in Schizophrenia Using a Multi-Domain Connectome Convolutional Neural Network
We exploit altered patterns in brain functional connectivity as features for
automatic discriminative analysis of neuropsychiatric patients. Deep learning
methods have been introduced to functional network classification only very
recently for fMRI, and the proposed architectures essentially focused on a
single type of connectivity measure. We propose a deep convolutional neural
network (CNN) framework for classification of electroencephalogram
(EEG)-derived brain connectome in schizophrenia (SZ). To capture complementary
aspects of disrupted connectivity in SZ, we explore combination of various
connectivity features consisting of time and frequency-domain metrics of
effective connectivity based on vector autoregressive model and partial
directed coherence, and complex network measures of network topology. We design
a novel multi-domain connectome CNN (MDC-CNN) based on a parallel ensemble of
1D and 2D CNNs to integrate the features from various domains and dimensions
using different fusion strategies. Hierarchical latent representations learned
by the multiple convolutional layers from EEG connectivity reveal apparent
group differences between SZ and healthy controls (HC). Results on a large
resting-state EEG dataset show that the proposed CNNs significantly outperform
traditional support vector machine classifiers. The MDC-CNN with combined
connectivity features further improves performance over single-domain CNNs
using individual features, achieving remarkable accuracy of with a
decision-level fusion. The proposed MDC-CNN by integrating information from
diverse brain connectivity descriptors is able to accurately discriminate SZ
from HC. The new framework is potentially useful for developing diagnostic
tools for SZ and other disorders.Comment: 15 pages, 9 figure
Deep Learning At Scale and At Ease
Recently, deep learning techniques have enjoyed success in various multimedia
applications, such as image classification and multi-modal data analysis. Large
deep learning models are developed for learning rich representations of complex
data. There are two challenges to overcome before deep learning can be widely
adopted in multimedia and other applications. One is usability, namely the
implementation of different models and training algorithms must be done by
non-experts without much effort especially when the model is large and complex.
The other is scalability, that is the deep learning system must be able to
provision for a huge demand of computing resources for training large models
with massive datasets. To address these two challenges, in this paper, we
design a distributed deep learning platform called SINGA which has an intuitive
programming model based on the common layer abstraction of deep learning
models. Good scalability is achieved through flexible distributed training
architecture and specific optimization techniques. SINGA runs on GPUs as well
as on CPUs, and we show that it outperforms many other state-of-the-art deep
learning systems. Our experience with developing and training deep learning
models for real-life multimedia applications in SINGA shows that the platform
is both usable and scalable.Comment: submitted to TOMM (under review
Topological based classification of paper domains using graph convolutional networks
The main approaches for node classification in graphs are information
propagation and the association of the class of the node with external
information. State of the art methods merge these approaches through Graph
Convolutional Networks. We here use the association of topological features of
the nodes with their class to predict this class. Moreover, combining
topological information with information propagation improves classification
accuracy on the standard CiteSeer and Cora paper classification task.
Topological features and information propagation produce results almost as good
as text-based classification, without no textual or content information. We
propose to represent the topology and information propagation through a GCN
with the neighboring training node classification as an input and the current
node classification as output. Such a formalism outperforms state of the art
methods
Scalable attribute-aware network embedding with locality
Adding attributes for nodes to network embedding helps to improve the ability
of the learned joint representation to depict features from topology and
attributes simultaneously. Recent research on the joint embedding has exhibited
a promising performance on a variety of tasks by jointly embedding the two
spaces. However, due to the indispensable requirement of globality based
information, present approaches contain a flaw of in-scalability. Here we
propose \emph{SANE}, a scalable attribute-aware network embedding algorithm
with locality, to learn the joint representation from topology and attributes.
By enforcing the alignment of a local linear relationship between each node and
its K-nearest neighbors in topology and attribute space, the joint embedding
representations are more informative comparing with a single representation
from topology or attributes alone. And we argue that the locality in
\emph{SANE} is the key to learning the joint representation at scale. By using
several real-world networks from diverse domains, We demonstrate the efficacy
of \emph{SANE} in performance and scalability aspect. Overall, for performance
on label classification, SANE successfully reaches up to the highest F1-score
on most datasets, and even closer to the baseline method that needs label
information as extra inputs, compared with other state-of-the-art joint
representation algorithms. What's more, \emph{SANE} has an up to 71.4\%
performance gain compared with the single topology-based algorithm. For
scalability, we have demonstrated the linearly time complexity of \emph{SANE}.
In addition, we intuitively observe that when the network size scales to
100,000 nodes, the "learning joint embedding" step of \emph{SANE} only takes
seconds
Multi-Label Image Recognition with Graph Convolutional Networks
The task of multi-label image recognition is to predict a set of object
labels that present in an image. As objects normally co-occur in an image, it
is desirable to model the label dependencies to improve the recognition
performance. To capture and explore such important dependencies, we propose a
multi-label classification model based on Graph Convolutional Network (GCN).
The model builds a directed graph over the object labels, where each node
(label) is represented by word embeddings of a label, and GCN is learned to map
this label graph into a set of inter-dependent object classifiers. These
classifiers are applied to the image descriptors extracted by another sub-net,
enabling the whole network to be end-to-end trainable. Furthermore, we propose
a novel re-weighted scheme to create an effective label correlation matrix to
guide information propagation among the nodes in GCN. Experiments on two
multi-label image recognition datasets show that our approach obviously
outperforms other existing state-of-the-art methods. In addition, visualization
analyses reveal that the classifiers learned by our model maintain meaningful
semantic topology.Comment: To appear at CVPR 2019 (Source codes have been released on
https://github.com/chenzhaomin123/ML_GCN
- …