76 research outputs found
CCG-augmented hierarchical phrase-based statistical machine translation
Augmenting Statistical Machine Translation (SMT) systems with syntactic information aims at improving translation quality. Hierarchical Phrase-Based (HPB) SMT takes a step toward incorporating syntax in Phrase-Based (PB) SMT by modelling one aspect of language syntax, namely the hierarchical structure of phrases. Syntax Augmented Machine Translation (SAMT) further incorporates syntactic information extracted using context free phrase structure grammar (CF-PSG) in the HPB SMT model. One of the main challenges facing CF-PSG-based augmentation approaches for SMT systems emerges from the difference in the definition of the constituent in CF-PSG and the âphraseâ in SMT systems, which hinders the ability of CF-PSG to express the syntactic function of many SMT phrases. Although the SAMT approach to solving this problem using âCCG-likeâ operators to combine constituent labels improves syntactic constraint coverage, it significantly increases their sparsity, which restricts translation and negatively affects its quality.
In this thesis, we address the problems of sparsity and limited coverage of syntactic constraints facing the CF-PSG-based syntax augmentation approaches for HPB SMT using Combinatory Cateogiral Grammar (CCG). We demonstrate that
CCGâs flexible structures and rich syntactic descriptors help to extract richer, more expressive and less sparse syntactic constraints with better coverage than CF-PSG,
which enables our CCG-augmented HPB system to outperform the SAMT system. We also try to soften the syntactic constraints imposed by CCG category nonterminal labels by extracting less fine-grained CCG-based labels. We demonstrate that CCG label simplification helps to significantly improve the performance of our CCG category HPB system. Finally, we identify the factors which limit the coverage of the syntactic constraints in our CCG-augmented HPB model. We then try to tackle these factors by extending the definition of the nonterminal label to be composed of a sequence of CCG categories and augmenting the glue grammar with CCG combinatory rules. We demonstrate that our extension approaches help to significantly increase the scope of the syntactic constraints applied in our CCG-augmented HPB model and achieve significant improvements over the HPB SMT baseline
A Syntactic Neural Model for General-Purpose Code Generation
We consider the problem of parsing natural language descriptions into source
code written in a general-purpose programming language like Python. Existing
data-driven methods treat this problem as a language generation task without
considering the underlying syntax of the target programming language. Informed
by previous work in semantic parsing, in this paper we propose a novel neural
architecture powered by a grammar model to explicitly capture the target syntax
as prior knowledge. Experiments find this an effective way to scale up to
generation of complex programs from natural language descriptions, achieving
state-of-the-art results that well outperform previous code generation and
semantic parsing approaches.Comment: To appear in ACL 201
Improving subject-verb agreement in SMT
Ensuring agreement between the subject and the main verb is crucial for the correctness of the information that a sentence conveys. While generating correct subject-verb agreement
is relatively straightforward in rule-based approaches to Machine Translation (RBMT), todayâs
leading statistical Machine Translation (SMT) systems often fail to generate correct subject-verb
agreements, especially when the target language is morphologically richer than the source language. The main problem is that one surface verb form in the source language corresponds to
many surface verb forms in the target language. To deal with subject-verb agreement we built a
hybrid SMT system that augments source verbs with extra linguistic information drawn from their
source-language context. This information, in the form of labels attached to verbs that indicate
person and number, creates a closer association between a verb from the source and a verb in the
target language. We used our preprocessing approach on English as source language and built an
SMT system for translation to French. In a range of experiments, the results show improvements
in translation quality for our augmented SMT system over a Moses baseline engine, on both automatic and manual evaluations, for the majority of cases where the subject-verb agreement was
previously incorrectly translated
Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics
This thesis is about the problem of compositionality in distributional
semantics. Distributional semantics presupposes that the meanings of words are
a function of their occurrences in textual contexts. It models words as
distributions over these contexts and represents them as vectors in high
dimensional spaces. The problem of compositionality for such models concerns
itself with how to produce representations for larger units of text by
composing the representations of smaller units of text.
This thesis focuses on a particular approach to this compositionality
problem, namely using the categorical framework developed by Coecke, Sadrzadeh,
and Clark, which combines syntactic analysis formalisms with distributional
semantic representations of meaning to produce syntactically motivated
composition operations. This thesis shows how this approach can be
theoretically extended and practically implemented to produce concrete
compositional distributional models of natural language semantics. It
furthermore demonstrates that such models can perform on par with, or better
than, other competing approaches in the field of natural language processing.
There are three principal contributions to computational linguistics in this
thesis. The first is to extend the DisCoCat framework on the syntactic front
and semantic front, incorporating a number of syntactic analysis formalisms and
providing learning procedures allowing for the generation of concrete
compositional distributional models. The second contribution is to evaluate the
models developed from the procedures presented here, showing that they
outperform other compositional distributional models present in the literature.
The third contribution is to show how using category theory to solve linguistic
problems forms a sound basis for research, illustrated by examples of work on
this topic, that also suggest directions for future research.Comment: DPhil Thesis, University of Oxford, Submitted and accepted in 201
A Computational Model of Syntactic Processing: Ambiguity Resolution from Interpretation
Syntactic ambiguity abounds in natural language, yet humans have no
difficulty coping with it. In fact, the process of ambiguity resolution is
almost always unconscious. But it is not infallible, however, as example 1
demonstrates.
1. The horse raced past the barn fell.
This sentence is perfectly grammatical, as is evident when it appears in the
following context:
2. Two horses were being shown off to a prospective buyer. One was raced past
a meadow. and the other was raced past a barn. ...
Grammatical yet unprocessable sentences such as 1 are called `garden-path
sentences.' Their existence provides an opportunity to investigate the human
sentence processing mechanism by studying how and when it fails. The aim of
this thesis is to construct a computational model of language understanding
which can predict processing difficulty. The data to be modeled are known
examples of garden path and non-garden path sentences, and other results from
psycholinguistics.
It is widely believed that there are two distinct loci of computation in
sentence processing: syntactic parsing and semantic interpretation. One
longstanding controversy is which of these two modules bears responsibility for
the immediate resolution of ambiguity. My claim is that it is the latter, and
that the syntactic processing module is a very simple device which blindly and
faithfully constructs all possible analyses for the sentence up to the current
point of processing. The interpretive module serves as a filter, occasionally
discarding certain of these analyses which it deems less appropriate for the
ongoing discourse than their competitors.
This document is divided into three parts. The first is introductory, and
reviews a selection of proposals from the sentence processing literature. The
second part explores a body of data which has been adduced in support of a
theory of structural preferences --- one that is inconsistent with the present
claim. I show how the current proposal can be specified to account for the
available data, and moreover to predict where structural preference theories
will go wrong. The third part is a theoretical investigation of how well the
proposed architecture can be realized using current conceptions of linguistic
competence. In it, I present a parsing algorithm and a meaning-based ambiguity
resolution method.Comment: 128 pages, LaTeX source compressed and uuencoded, figures separate
macros: rotate.sty, lingmacros.sty, psfig.tex. Dissertation, Computer and
Information Science Dept., October 199
CLiFF Notes: Research in the Language Information and Computation Laboratory of The University of Pennsylvania
This report takes its name from the Computational Linguistics Feedback Forum (CLIFF), an informal discussion group for students and faculty. However the scope of the research covered in this report is broader than the title might suggest; this is the yearly report of the LINC Lab, the Language, Information and Computation Laboratory of the University of Pennsylvania. It may at first be hard to see the threads that bind together the work presented here, work by faculty, graduate students and postdocs in the Computer Science, Psychology, and Linguistics Departments, and the Institute for Research in Cognitive Science. It includes prototypical Natural Language fields such as: Combinatorial Categorial Grammars, Tree Adjoining Grammars, syntactic parsing and the syntax-semantics interface; but it extends to statistical methods, plan inference, instruction understanding, intonation, causal reasoning, free word order languages, geometric reasoning, medical informatics, connectionism, and language acquisition. With 48 individual contributors and six projects represented, this is the largest LINC Lab collection to date, and the most diverse
- âŠ