6,208 research outputs found
Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks
We propose a distance supervised relation extraction approach for
long-tailed, imbalanced data which is prevalent in real-world settings. Here,
the challenge is to learn accurate "few-shot" models for classes existing at
the tail of the class distribution, for which little data is available.
Inspired by the rich semantic correlations between classes at the long tail and
those at the head, we take advantage of the knowledge from data-rich classes at
the head of the distribution to boost the performance of the data-poor classes
at the tail. First, we propose to leverage implicit relational knowledge among
class labels from knowledge graph embeddings and learn explicit relational
knowledge using graph convolution networks. Second, we integrate that
relational knowledge into relation extraction model by coarse-to-fine
knowledge-aware attention mechanism. We demonstrate our results for a
large-scale benchmark dataset which show that our approach significantly
outperforms other baselines, especially for long-tail relations.Comment: To be published in NAACL 201
Probabilistic Label Relation Graphs with Ising Models
We consider classification problems in which the label space has structure. A
common example is hierarchical label spaces, corresponding to the case where
one label subsumes another (e.g., animal subsumes dog). But labels can also be
mutually exclusive (e.g., dog vs cat) or unrelated (e.g., furry, carnivore). To
jointly model hierarchy and exclusion relations, the notion of a HEX (hierarchy
and exclusion) graph was introduced in [7]. This combined a conditional random
field (CRF) with a deep neural network (DNN), resulting in state of the art
results when applied to visual object classification problems where the
training labels were drawn from different levels of the ImageNet hierarchy
(e.g., an image might be labeled with the basic level category "dog", rather
than the more specific label "husky"). In this paper, we extend the HEX model
to allow for soft or probabilistic relations between labels, which is useful
when there is uncertainty about the relationship between two labels (e.g., an
antelope is "sort of" furry, but not to the same degree as a grizzly bear). We
call our new model pHEX, for probabilistic HEX. We show that the pHEX graph can
be converted to an Ising model, which allows us to use existing off-the-shelf
inference methods (in contrast to the HEX method, which needed specialized
inference algorithms). Experimental results show significant improvements in a
number of large-scale visual object classification tasks, outperforming the
previous HEX model.Comment: International Conference on Computer Vision (2015
Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
Many interesting problems in machine learning are being revisited with new
deep learning tools. For graph-based semisupervised learning, a recent
important development is graph convolutional networks (GCNs), which nicely
integrate local vertex features and graph topology in the convolutional layers.
Although the GCN model compares favorably with other state-of-the-art methods,
its mechanisms are not clear and it still requires a considerable amount of
labeled data for validation and model selection. In this paper, we develop
deeper insights into the GCN model and address its fundamental limits. First,
we show that the graph convolution of the GCN model is actually a special form
of Laplacian smoothing, which is the key reason why GCNs work, but it also
brings potential concerns of over-smoothing with many convolutional layers.
Second, to overcome the limits of the GCN model with shallow architectures, we
propose both co-training and self-training approaches to train GCNs. Our
approaches significantly improve GCNs in learning with very few labels, and
exempt them from requiring additional labels for validation. Extensive
experiments on benchmarks have verified our theory and proposals.Comment: AAAI-2018 Oral Presentatio
- …