15,036 research outputs found
Human-Guided Learning of Column Networks: Augmenting Deep Learning with Advice
Recently, deep models have been successfully applied in several applications,
especially with low-level representations. However, sparse, noisy samples and
structured domains (with multiple objects and interactions) are some of the
open challenges in most deep models. Column Networks, a deep architecture, can
succinctly capture such domain structure and interactions, but may still be
prone to sub-optimal learning from sparse and noisy samples. Inspired by the
success of human-advice guided learning in AI, especially in data-scarce
domains, we propose Knowledge-augmented Column Networks that leverage human
advice/knowledge for better learning with noisy/sparse samples. Our experiments
demonstrate that our approach leads to either superior overall performance or
faster convergence (i.e., both effective and efficient).Comment: Under Review at 'Machine Learning Journal' (MLJ
Learning Structured Inference Neural Networks with Label Relations
Images of scenes have various objects as well as abundant attributes, and
diverse levels of visual categorization are possible. A natural image could be
assigned with fine-grained labels that describe major components,
coarse-grained labels that depict high level abstraction or a set of labels
that reveal attributes. Such categorization at different concept layers can be
modeled with label graphs encoding label information. In this paper, we exploit
this rich information with a state-of-art deep learning framework, and propose
a generic structured model that leverages diverse label relations to improve
image classification performance. Our approach employs a novel stacked label
prediction neural network, capturing both inter-level and intra-level label
semantics. We evaluate our method on benchmark image datasets, and empirical
results illustrate the efficacy of our model.Comment: Conference on Computer Vision and Pattern Recognition(CVPR) 201
COSINE: Compressive Network Embedding on Large-scale Information Networks
There is recently a surge in approaches that learn low-dimensional embeddings
of nodes in networks. As there are many large-scale real-world networks, it's
inefficient for existing approaches to store amounts of parameters in memory
and update them edge after edge. With the knowledge that nodes having similar
neighborhood will be close to each other in embedding space, we propose COSINE
(COmpresSIve NE) algorithm which reduces the memory footprint and accelerates
the training process by parameters sharing among similar nodes. COSINE applies
graph partitioning algorithms to networks and builds parameter sharing
dependency of nodes based on the result of partitioning. With parameters
sharing among similar nodes, COSINE injects prior knowledge about higher
structural information into training process which makes network embedding more
efficient and effective. COSINE can be applied to any embedding lookup method
and learn high-quality embeddings with limited memory and shorter training
time. We conduct experiments of multi-label classification and link prediction,
where baselines and our model have the same memory usage. Experimental results
show that COSINE gives baselines up to 23% increase on classification and up to
25% increase on link prediction. Moreover, time of all representation learning
methods using COSINE decreases from 30% to 70%
A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications
Graph is an important data representation which appears in a wide diversity
of real-world scenarios. Effective graph analytics provides users a deeper
understanding of what is behind the data, and thus can benefit a lot of useful
applications such as node classification, node recommendation, link prediction,
etc. However, most graph analytics methods suffer the high computation and
space cost. Graph embedding is an effective yet efficient way to solve the
graph analytics problem. It converts the graph data into a low dimensional
space in which the graph structural information and graph properties are
maximally preserved. In this survey, we conduct a comprehensive review of the
literature in graph embedding. We first introduce the formal definition of
graph embedding as well as the related concepts. After that, we propose two
taxonomies of graph embedding which correspond to what challenges exist in
different graph embedding problem settings and how the existing work address
these challenges in their solutions. Finally, we summarize the applications
that graph embedding enables and suggest four promising future research
directions in terms of computation efficiency, problem settings, techniques and
application scenarios.Comment: A 20-page comprehensive survey of graph/network embedding for over
150+ papers till year 2018. It provides systematic categorization of
problems, techniques and applications. Accepted by IEEE Transactions on
Knowledge and Data Engineering (TKDE). Comments and suggestions are welcomed
for continuously improving this surve
Low-Rank Deep Convolutional Neural Network for Multi-Task Learning
In this paper, we propose a novel multi-task learning method based on the
deep convolutional network. The proposed deep network has four convolutional
layers, three max-pooling layers, and two parallel fully connected layers. To
adjust the deep network to multi-task learning problem, we propose to learn a
low-rank deep network so that the relation among different tasks can be
explored. We proposed to minimize the number of independent parameter rows of
one fully connected layer to explore the relations among different tasks, which
is measured by the nuclear norm of the parameter of one fully connected layer,
and seek a low-rank parameter matrix. Meanwhile, we also propose to regularize
another fully connected layer by sparsity penalty, so that the useful features
learned by the lower layers can be selected. The learning problem is solved by
an iterative algorithm based on gradient descent and back-propagation
algorithms. The proposed algorithm is evaluated over benchmark data sets of
multiple face attribute prediction, multi-task natural language processing, and
joint economics index predictions. The evaluation results show the advantage of
the low-rank deep CNN model over multi-task problems
Multi-Target Prediction: A Unifying View on Problems and Methods
Multi-target prediction (MTP) is concerned with the simultaneous prediction
of multiple target variables of diverse type. Due to its enormous application
potential, it has developed into an active and rapidly expanding research field
that combines several subfields of machine learning, including multivariate
regression, multi-label classification, multi-task learning, dyadic prediction,
zero-shot learning, network inference, and matrix completion. In this paper, we
present a unifying view on MTP problems and methods. First, we formally discuss
commonalities and differences between existing MTP problems. To this end, we
introduce a general framework that covers the above subfields as special cases.
As a second contribution, we provide a structured overview of MTP methods. This
is accomplished by identifying a number of key properties, which distinguish
such methods and determine their suitability for different types of problems.
Finally, we also discuss a few challenges for future research
Attentional Multilabel Learning over Graphs: A Message Passing Approach
We address a largely open problem of multilabel classification over graphs.
Unlike traditional vector input, a graph has rich variable-size substructures
which are related to the labels in some ways. We believe that uncovering these
relations might hold the key to classification performance and explainability.
We introduce GAML (Graph Attentional Multi-Label learning), a novel graph
neural network that can handle this problem effectively. GAML regards labels as
auxiliary nodes and models them in conjunction with the input graph. By
applying message passing and attention mechanisms to both the label nodes and
the input nodes iteratively, GAML can capture the relations between the labels
and the input subgraphs at various resolution scales. Moreover, our model can
take advantage of explicit label dependencies. It also scales linearly with the
number of labels and graph size thanks to our proposed hierarchical attention.
We evaluate GAML on an extensive set of experiments with both graph-structured
inputs and classical unstructured inputs. The results show that GAML
significantly outperforms other competing methods. Importantly, GAML enables
intuitive visualizations for better understanding of the label-substructure
relations and explanation of the model behaviors
Dynamic-structured Semantic Propagation Network
Semantic concept hierarchy is still under-explored for semantic segmentation
due to the inefficiency and complicated optimization of incorporating
structural inference into dense prediction. This lack of modeling semantic
correlations also makes prior works must tune highly-specified models for each
task due to the label discrepancy across datasets. It severely limits the
generalization capability of segmentation models for open set concept
vocabulary and annotation utilization. In this paper, we propose a
Dynamic-Structured Semantic Propagation Network (DSSPN) that builds a semantic
neuron graph by explicitly incorporating the semantic concept hierarchy into
network construction. Each neuron represents the instantiated module for
recognizing a specific type of entity such as a super-class (e.g. food) or a
specific concept (e.g. pizza). During training, DSSPN performs the
dynamic-structured neuron computation graph by only activating a sub-graph of
neurons for each image in a principled way. A dense semantic-enhanced neural
block is proposed to propagate the learned knowledge of all ancestor neurons
into each fine-grained child neuron for feature evolving. Another merit of such
semantic explainable structure is the ability of learning a unified model
concurrently on diverse datasets by selectively activating different neuron
sub-graphs for each annotation at each step. Extensive experiments on four
public semantic segmentation datasets (i.e. ADE20K, COCO-Stuff, Cityscape and
Mapillary) demonstrate the superiority of our DSSPN over state-of-the-art
segmentation models. Moreoever, we demonstrate a universal segmentation model
that is jointly trained on diverse datasets can surpass the performance of the
common fine-tuning scheme for exploiting multiple domain knowledge.Comment: CVPR 201
Using Context Information to Enhance Simple Question Answering
With the rapid development of knowledge bases(KBs),question
answering(QA)based on KBs has become a hot research issue. In this paper,we
propose two frameworks(i.e.,pipeline framework,an end-to-end framework)to focus
answering single-relation factoid question. In both of two frameworks,we study
the effect of context information on the quality of QA,such as the entity's
notable type,out-degree. In the end-to-end framework,we combine char-level
encoding and self-attention mechanisms,using weight sharing and multi-task
strategies to enhance the accuracy of QA. Experimental results show that
context information can get better results of simple QA whether it is the
pipeline framework or the end-to-end framework. In addition,we find that the
end-to-end framework achieves results competitive with state-of-the-art
approaches in terms of accuracy and take much shorter time than them.Comment: under review World Wide Web Journa
Learning multi-faceted representations of individuals from heterogeneous evidence using neural networks
Inferring latent attributes of people online is an important social computing
task, but requires integrating the many heterogeneous sources of information
available on the web. We propose learning individual representations of people
using neural nets to integrate rich linguistic and network evidence gathered
from social media. The algorithm is able to combine diverse cues, such as the
text a person writes, their attributes (e.g. gender, employer, education,
location) and social relations to other people. We show that by integrating
both textual and network evidence, these representations offer improved
performance at four important tasks in social media inference on Twitter:
predicting (1) gender, (2) occupation, (3) location, and (4) friendships for
users. Our approach scales to large datasets and the learned representations
can be used as general features in and have the potential to benefit a large
number of downstream tasks including link prediction, community detection, or
probabilistic reasoning over social networks
- …