93 research outputs found
Weakly-supervised Neural Semantic Parsing with a Generative Ranker
Weakly-supervised semantic parsers are trained on utterance-denotation pairs,
treating logical forms as latent. The task is challenging due to the large
search space and spuriousness of logical forms. In this paper we introduce a
neural parser-ranker system for weakly-supervised semantic parsing. The parser
generates candidate tree-structured logical forms from utterances using clues
of denotations. These candidates are then ranked based on two criterion: their
likelihood of executing to the correct denotation, and their agreement with the
utterance semantics. We present a scheduled training procedure to balance the
contribution of the two objectives. Furthermore, we propose to use a neurally
encoded lexicon to inject prior domain knowledge to the model. Experiments on
three Freebase datasets demonstrate the effectiveness of our semantic parser,
achieving results within the state-of-the-art range.Comment: In EMNLP-CoNLL 201
Lifecycle of neural semantic parsing
Humans are born with the ability to learn to perceive, comprehend and communicate
with language. Computing machines, on the other hand, only understand programming
languages. To bridge the gap between humans and computers, deep semantic parsers
convert natural language utterances into machine-understandable logical forms. The
technique has a wide range of applications ranging from spoken dialogue systems and
natural language interfaces. This thesis focuses on neural network-based semantic
parsing.
Traditional semantic parsers function with a domain-specific grammar that pairs
utterances and logical forms, and parse with a CKY-like algorithm in polynomial
time. Recent advances in neural semantic parsing reformulate the task as a sequence-to-
sequence learning problem. Neural semantic parsers parse a sentence in linear
time, and reduce the need for domain-specific assumptions, grammar learning, and
extensive feature engineering. But this modeling flexibility comes at a cost since
it is no longer possible to interpret how meaning composition is performed, given
that logical forms are structured objects (trees or graphs). Such knowledge plays
a critical role in understanding modeling limitations so as to build better semantic
parsers. Moreover, the sequence-to-sequence learning problem is fairly unconstrained,
both in terms of the possible derivations to consider and in terms of the target logical
forms which can be ill-formed or unexecutable. The first contribution of this thesis is
an improved neural semantic parser, which produces syntactically valid logical forms
following a transition system and grammar constrains. The transition system integrates
the generation of domain-general (i.e., valid tree-structures and language-specific predicates)
and domain-specific aspects (i.e., domain-specific predicates and entities) in a unified
way. The model employs various neural attention mechanisms to handle mismatches
between natural language and formal language—a central challenge in semantic parsing.
Training data to semantic parsers typically consists of utterances paired with logical
forms. Another challenge of semantic parsing concerns the annotation of logical forms,
which is labor-intensive. To write down the correct logical form of an utterance, one
not only needs to have expertise in the semantic formalism, but also has to ensure the
logical form matches the utterance semantics. We tackle this challenge in two ways.
On the one hand, we extend the neural semantic parser to a weakly-supervised setting
within a parser-ranker framework. The weakly-supervised setup uses training data
of utterance-denotation (e.g., question-answer) pairs, which are much easier to obtain
and therefore allow to scale semantic parsers to complex domains. Our framework
combines the advantages of conventional weakly-supervised semantic parsers and neural
semantic parsing. Candidate logical forms are generated by a neural decoder and
subsequently scored by a ranking component. We present methods to efficiently search
for candidate logical forms which involve spurious ambiguity—some logical forms do
not match utterance semantics but coincidentally execute to the correct denotation.
They should be excluded from training.
On the other hand, we focus on how to quickly engineer a practical neural semantic
parser for closed domains, by directly reducing the annotation difficulty of utterance-logical
form pairs. We develop an interface for efficiently collecting compositional
utterance-logical form pairs and then leverage the data collection method to train neural
semantic parsers. Our method provides an end-to-end solution for closed-domain
semantic parsing given only an ontology. We also extend the end-to-end solution to
handle sequential utterances simulating a non-interactive user session. Specifically,
the data collection interface is modified to collect utterance sequences which exhibit
various co-reference patterns. Then the neural semantic parser is extended to parse
context-dependent utterances.
In summary, this thesis covers the lifecycle of designing a neural semantic parser:
from model design (i.e., how to model a neural semantic parser with an appropriate
inductive bias), training (i.e., how to perform fully supervised and weakly supervised
training for a neural semantic parser) to engineering (i.e., how to build a neural semantic
parser from a domain ontology)
Learning an Executable Neural Semantic Parser
This paper describes a neural semantic parser that maps natural language
utterances onto logical forms which can be executed against a task-specific
environment, such as a knowledge base or a database, to produce a response. The
parser generates tree-structured logical forms with a transition-based approach
which combines a generic tree-generation algorithm with domain-general
operations defined by the logical language. The generation process is modeled
by structured recurrent neural networks, which provide a rich encoding of the
sentential context and generation history for making predictions. To tackle
mismatches between natural language and logical form tokens, various attention
mechanisms are explored. Finally, we consider different training settings for
the neural semantic parser, including a fully supervised training where
annotated logical forms are given, weakly-supervised training where denotations
are provided, and distant supervision where only unlabeled sentences and a
knowledge base are available. Experiments across a wide range of datasets
demonstrate the effectiveness of our parser.Comment: In Journal of Computational Linguistic
Merging Weak and Active Supervision for Semantic Parsing
A semantic parser maps natural language commands (NLs) from the users to
executable meaning representations (MRs), which are later executed in certain
environment to obtain user-desired results. The fully-supervised training of
such parser requires NL/MR pairs, annotated by domain experts, which makes them
expensive to collect. However, weakly-supervised semantic parsers are learnt
only from pairs of NL and expected execution results, leaving the MRs latent.
While weak supervision is cheaper to acquire, learning from this input poses
difficulties. It demands that parsers search a large space with a very weak
learning signal and it is hard to avoid spurious MRs that achieve the correct
answer in the wrong way. These factors lead to a performance gap between
parsers trained in weakly- and fully-supervised setting. To bridge this gap, we
examine the intersection between weak supervision and active learning, which
allows the learner to actively select examples and query for manual annotations
as extra supervision to improve the model trained under weak supervision. We
study different active learning heuristics for selecting examples to query, and
various forms of extra supervision for such queries. We evaluate the
effectiveness of our method on two different datasets. Experiments on the
WikiSQL show that by annotating only 1.8% of examples, we improve over a
state-of-the-art weakly-supervised baseline by 6.4%, achieving an accuracy of
79.0%, which is only 1.3% away from the model trained with full supervision.
Experiments on WikiTableQuestions with human annotators show that our method
can improve the performance with only 100 active queries, especially for
weakly-supervised parsers learnt from a cold start.Comment: AAAI 2020 Main Track [Oral] (To appear
- …