68 research outputs found
Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension
Reading comprehension is a challenging task in natural language processing
and requires a set of skills to be solved. While current approaches focus on
solving the task as a whole, in this paper, we propose to use a neural network
`skill' transfer approach. We transfer knowledge from several lower-level
language tasks (skills) including textual entailment, named entity recognition,
paraphrase detection and question type classification into the reading
comprehension model.
We conduct an empirical evaluation and show that transferring language skill
knowledge leads to significant improvements for the task with much fewer steps
compared to the baseline model. We also show that the skill transfer approach
is effective even with small amounts of training data. Another finding of this
work is that using token-wise deep label supervision for text classification
improves the performance of transfer learning
Learning and evaluating the content and structure of a term taxonomy
Journal ArticleIn this paper, we describe a weakly supervised bootstrapping algorithm that reads Web texts and learns taxonomy terms. The bootstrapping algorithm starts with two seed words (a seed hypernym (Root concept) and a seed hyponym) that are inserted into a doubly anchored hyponym pattern. In alternating rounds, the algorithm learns new hyponym terms and new hypernym terms that are subordinate to the Root concept. We conducted an extensive evaluation with human annotators to evaluate the learned hyponym and hypernym terms for two categories: animals and people
Semantic class learning from the web with hyponym pattern linkage graphs
Journal ArticleWe present a novel approach to weakly supervised semantic class learning from the web, using a single powerful hyponym pattern combined with graph structures, which capture two properties associated with pattern-based extractions: popularity and productivity. Intuitively, a candidate is popular if it was discovered many times by other instances in the hyponym pattern. A candidate is productive if it frequently leads to the discovery of other instances. Together, these two measures capture not only frequency of occurrence, but also cross-checking that the candidate occurs both near the class name and near other class members. We developed two algorithms that begin with just a class name and one seed instance and then automatically generate a ranked list of new class instances. We conducted experiments on four semantic classes and consistently achieved high accuracies
OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]
Extraction of missing attribute values is to find values describing an
attribute of interest from a free text input. Most past related work on
extraction of missing attribute values work with a closed world assumption with
the possible set of values known beforehand, or use dictionaries of values and
hand-crafted features. How can we discover new attribute values that we have
never seen before? Can we do this with limited human annotation or supervision?
We study this problem in the context of product catalogs that often have
missing values for many attributes of interest.
In this work, we leverage product profile information such as titles and
descriptions to discover missing values of product attributes. We develop a
novel deep tagging model OpenTag for this extraction problem with the following
contributions: (1) we formalize the problem as a sequence tagging task, and
propose a joint model exploiting recurrent neural networks (specifically,
bidirectional LSTM) to capture context and semantics, and Conditional Random
Fields (CRF) to enforce tagging consistency, (2) we develop a novel attention
mechanism to provide interpretable explanation for our model's decisions, (3)
we propose a novel sampling strategy exploring active learning to reduce the
burden of human annotation. OpenTag does not use any dictionary or hand-crafted
features as in prior works. Extensive experiments in real-life datasets in
different domains show that OpenTag with our active learning strategy discovers
new attribute values from as few as 150 annotated samples (reduction in 3.3x
amount of annotation effort) with a high F-score of 83%, outperforming
state-of-the-art models.Comment: Proceedings of the 24th ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining, London, UK, August 19-23, 201
- …