5 research outputs found
Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning
Neural semantic parsing has achieved impressive results in recent years, yet
its success relies on the availability of large amounts of supervised data. Our
goal is to learn a neural semantic parser when only prior knowledge about a
limited number of simple rules is available, without access to either annotated
programs or execution results. Our approach is initialized by rules, and
improved in a back-translation paradigm using generated question-program pairs
from the semantic parser and the question generator. A phrase table with
frequent mapping patterns is automatically derived, also updated as training
progresses, to measure the quality of generated instances. We train the model
with model-agnostic meta-learning to guarantee the accuracy and stability on
examples covered by rules, and meanwhile acquire the versatility to generalize
well on examples uncovered by rules. Results on three benchmark datasets with
different domains and programs show that our approach incrementally improves
the accuracy. On WikiSQL, our best model is comparable to the SOTA system
learned from denotations
Recommended from our members
Using domain specific language and sequence to sequence models as a hybrid framework for a natural language interface to a database solution
This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe aim of this project is to provide a new approach to solving the problem of
converting natural language into a language capable of querying a database or data
repository. This problem has been around for a while, in the 1970's the US Navy
developed a solution called LADDER and since then there have been an array of
solutions, approaches and tweaks that have kept the research community busy. The
introduction of electronic assistants into the smart phone in 2010 has given new
impetus to this problem.
With the increasingly pervasive nature of data and its ever expanding use to answer
questions within business science, medicine extracting data is becoming more important.
The idea behind this project is to make data more democratised by allowing access to it
without the need for specialist languages. The performance and reliability of converting
natural language into structured query language can be problematic in handling nuances
that are prevalent in natural language. Relational databases are not designed to understand
language nuance.
This project introduces the following components as part of a holistic approach to improving
the conversion of a natural language statement into a language capable of querying a data
repository.
● The idea proposed in this project combines the use of sequence to sequence models
in conjunction with the natural language part of speech technologies and domain
specific languages to convert natural language queries into SQL. The approach
being proposed by this chapter is to use natural language processing to perform an
initial shallow pass of the incoming query and then use Google's Tensor Flow to
refine the query with the use of a sequence to sequence model.
● This thesis is also proposing to use a Domain Specific Language (DSL) as part of the
conversion process. The use of the DSL has the potential to allow the natural
language query to be translated into more than just an SQL statement, but any query
language such as NoSQL or XQuery
Lifecycle of neural semantic parsing
Humans are born with the ability to learn to perceive, comprehend and communicate
with language. Computing machines, on the other hand, only understand programming
languages. To bridge the gap between humans and computers, deep semantic parsers
convert natural language utterances into machine-understandable logical forms. The
technique has a wide range of applications ranging from spoken dialogue systems and
natural language interfaces. This thesis focuses on neural network-based semantic
parsing.
Traditional semantic parsers function with a domain-specific grammar that pairs
utterances and logical forms, and parse with a CKY-like algorithm in polynomial
time. Recent advances in neural semantic parsing reformulate the task as a sequence-to-
sequence learning problem. Neural semantic parsers parse a sentence in linear
time, and reduce the need for domain-specific assumptions, grammar learning, and
extensive feature engineering. But this modeling flexibility comes at a cost since
it is no longer possible to interpret how meaning composition is performed, given
that logical forms are structured objects (trees or graphs). Such knowledge plays
a critical role in understanding modeling limitations so as to build better semantic
parsers. Moreover, the sequence-to-sequence learning problem is fairly unconstrained,
both in terms of the possible derivations to consider and in terms of the target logical
forms which can be ill-formed or unexecutable. The first contribution of this thesis is
an improved neural semantic parser, which produces syntactically valid logical forms
following a transition system and grammar constrains. The transition system integrates
the generation of domain-general (i.e., valid tree-structures and language-specific predicates)
and domain-specific aspects (i.e., domain-specific predicates and entities) in a unified
way. The model employs various neural attention mechanisms to handle mismatches
between natural language and formal language—a central challenge in semantic parsing.
Training data to semantic parsers typically consists of utterances paired with logical
forms. Another challenge of semantic parsing concerns the annotation of logical forms,
which is labor-intensive. To write down the correct logical form of an utterance, one
not only needs to have expertise in the semantic formalism, but also has to ensure the
logical form matches the utterance semantics. We tackle this challenge in two ways.
On the one hand, we extend the neural semantic parser to a weakly-supervised setting
within a parser-ranker framework. The weakly-supervised setup uses training data
of utterance-denotation (e.g., question-answer) pairs, which are much easier to obtain
and therefore allow to scale semantic parsers to complex domains. Our framework
combines the advantages of conventional weakly-supervised semantic parsers and neural
semantic parsing. Candidate logical forms are generated by a neural decoder and
subsequently scored by a ranking component. We present methods to efficiently search
for candidate logical forms which involve spurious ambiguity—some logical forms do
not match utterance semantics but coincidentally execute to the correct denotation.
They should be excluded from training.
On the other hand, we focus on how to quickly engineer a practical neural semantic
parser for closed domains, by directly reducing the annotation difficulty of utterance-logical
form pairs. We develop an interface for efficiently collecting compositional
utterance-logical form pairs and then leverage the data collection method to train neural
semantic parsers. Our method provides an end-to-end solution for closed-domain
semantic parsing given only an ontology. We also extend the end-to-end solution to
handle sequential utterances simulating a non-interactive user session. Specifically,
the data collection interface is modified to collect utterance sequences which exhibit
various co-reference patterns. Then the neural semantic parser is extended to parse
context-dependent utterances.
In summary, this thesis covers the lifecycle of designing a neural semantic parser:
from model design (i.e., how to model a neural semantic parser with an appropriate
inductive bias), training (i.e., how to perform fully supervised and weakly supervised
training for a neural semantic parser) to engineering (i.e., how to build a neural semantic
parser from a domain ontology)