2,939 research outputs found
A Survey of Word Reordering in Statistical Machine Translation: Computational Models and Language Phenomena
Word reordering is one of the most difficult aspects of statistical machine
translation (SMT), and an important factor of its quality and efficiency.
Despite the vast amount of research published to date, the interest of the
community in this problem has not decreased, and no single method appears to be
strongly dominant across language pairs. Instead, the choice of the optimal
approach for a new translation task still seems to be mostly driven by
empirical trials. To orientate the reader in this vast and complex research
area, we present a comprehensive survey of word reordering viewed as a
statistical modeling challenge and as a natural language phenomenon. The survey
describes in detail how word reordering is modeled within different
string-based and tree-based SMT frameworks and as a stand-alone task, including
systematic overviews of the literature in advanced reordering modeling. We then
question why some approaches are more successful than others in different
language pairs. We argue that, besides measuring the amount of reordering, it
is important to understand which kinds of reordering occur in a given language
pair. To this end, we conduct a qualitative analysis of word reordering
phenomena in a diverse sample of language pairs, based on a large collection of
linguistic knowledge. Empirical results in the SMT literature are shown to
support the hypothesis that a few linguistic facts can be very useful to
anticipate the reordering characteristics of a language pair and to select the
SMT framework that best suits them.Comment: 44 pages, to appear in Computational Linguistic
Improving Compositional Generalization with Latent Structure and Data Augmentation
Generic unstructured neural networks have been shown to struggle on
out-of-distribution compositional generalization. Compositional data
augmentation via example recombination has transferred some prior knowledge
about compositionality to such black-box neural models for several semantic
parsing tasks, but this often required task-specific engineering or provided
limited gains.
We present a more powerful data recombination method using a model called
Compositional Structure Learner (CSL). CSL is a generative model with a
quasi-synchronous context-free grammar backbone, which we induce from the
training data. We sample recombined examples from CSL and add them to the
fine-tuning data of a pre-trained sequence-to-sequence model (T5). This
procedure effectively transfers most of CSL's compositional bias to T5 for
diagnostic tasks, and results in a model even stronger than a T5-CSL ensemble
on two real world compositional generalization tasks. This results in new
state-of-the-art performance for these challenging semantic parsing tasks
requiring generalization to both natural language variation and novel
compositions of elements.Comment: NAACL 202
Compositional Generalisation with Structured Reordering and Fertility Layers
Seq2seq models have been shown to struggle with compositional generalisation,
i.e. generalising to new and potentially more complex structures than seen
during training. Taking inspiration from grammar-based models that excel at
compositional generalisation, we present a flexible end-to-end differentiable
neural model that composes two structural operations: a fertility step, which
we introduce in this work, and a reordering step based on previous work (Wang
et al., 2021). Our model outperforms seq2seq models by a wide margin on
challenging compositional splits of realistic semantic parsing tasks that
require generalisation to longer examples. It also compares favourably to other
models targeting compositional generalisation
Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands
To understand diverse natural language commands, virtual assistants today are
trained with numerous labor-intensive, manually annotated sentences. This paper
presents a methodology and the Genie toolkit that can handle new compound
commands with significantly less manual effort. We advocate formalizing the
capability of virtual assistants with a Virtual Assistant Programming Language
(VAPL) and using a neural semantic parser to translate natural language into
VAPL code. Genie needs only a small realistic set of input sentences for
validating the neural model. Developers write templates to synthesize data;
Genie uses crowdsourced paraphrases and data augmentation, along with the
synthesized data, to train a semantic parser. We also propose design principles
that make VAPL languages amenable to natural language translation. We apply
these principles to revise ThingTalk, the language used by the Almond virtual
assistant. We use Genie to build the first semantic parser that can support
compound virtual assistants commands with unquoted free-form parameters. Genie
achieves a 62% accuracy on realistic user inputs. We demonstrate Genie's
generality by showing a 19% and 31% improvement over the previous state of the
art on a music skill, aggregate functions, and access control.Comment: To appear in PLDI 201
- …