18 research outputs found
Recommended from our members
Hindi Complex Predicates: Linguistic and Computational Approaches
Complex predicates that comprise of a noun and verb e.g. yaad kar 'memory do; remember' are a productive class of multi-words in Hindi. In this thesis, we examine the challenges of identification and representation for these complex predicates in Hindi. We design and implement their representation in a lexical semantic resource as well as in lexicalized computational grammars. As productive multi-word predicates, their accurate identification is a necessity for natural language processing applications. We use a combination of linguistic and computational approaches to address these challenges. We use these methods to demonstrate the semi-automatic creation of subcategorization frames for Hindi and the development of classes for nominal predicates. Finally, we demonstrate how linguistic features and computational tools can be used in tandem to automatically identify complex predicates from unseen text
Proceedings
Proceedings of the Ninth International Workshop
on Treebanks and Linguistic Theories.
Editors: Markus Dickinson, Kaili Müürisep and Marco Passarotti.
NEALT Proceedings Series, Vol. 9 (2010), 268 pages.
© 2010 The editors and contributors.
Published by
Northern European Association for Language
Technology (NEALT)
http://omilia.uio.no/nealt .
Electronically published at
Tartu University Library (Estonia)
http://hdl.handle.net/10062/15891
Integrating source-language context into log-linear models of statistical machine translation
The translation features typically used in state-of-the-art statistical machine translation (SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear phrase-based SMT (PB-SMT) and hierarchical PB-SMT (HPB-SMT), and can positively
influence the weighting and selection of target phrases, and thus improve translation quality. In this thesis we present novel approaches to incorporate source-language contextual modelling into the state-of-the-art SMT models in order to enhance the quality of lexical selection. We investigate the effectiveness of use of a range of contextual features, including lexical features of neighbouring words, part-of-speech tags, supertags, sentence-similarity features, dependency information, and semantic roles. We explored a series of language pairs featuring typologically different languages, and examined the scalability of our research to larger amounts of training data.
While our results are mixed across feature selections, language pairs, and learning curves, we observe that including contextual features of the source sentence
in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in
combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, supertag features in English-to-Chinese translation, or combination of supertag and lexical features in English-to-Dutch subtitle
translation. Furthermore, we investigate the applicability of our lexical contextual model in another closely related NLP problem, namely machine transliteration
Broad-coverage model of prediction in human sentence processing
The aim of this thesis is to design and implement a cognitively plausible theory
of sentence processing which incorporates a mechanism for modeling a prediction
and verification process in human language understanding, and to evaluate the validity
of this model on specific psycholinguistic phenomena as well as on broad-coverage,
naturally occurring text.
Modeling prediction is a timely and relevant contribution to the field because recent
experimental evidence suggests that humans predict upcoming structure or lexemes
during sentence processing. However, none of the current sentence processing theories
capture prediction explicitly. This thesis proposes a novel model of incremental
sentence processing that offers an explicit prediction and verification mechanism.
In evaluating the proposed model, this thesis also makes a methodological contribution.
The design and evaluation of current sentence processing theories are usually
based exclusively on experimental results from individual psycholinguistic experiments
on specific linguistic structures. However, a theory of language processing in
humans should not only work in an experimentally designed environment, but should
also have explanatory power for naturally occurring language.
This thesis first shows that the Dundee corpus, an eye-tracking corpus of newspaper
text, constitutes a valuable additional resource for testing sentence processing theories.
I demonstrate that a benchmark processing effect (the subject/object relative clause
asymmetry) can be detected in this data set (Chapter 4). I then evaluate two existing
theories of sentence processing, Surprisal and Dependency Locality Theory (DLT),
on the full Dundee corpus. This constitutes the first broad-coverage comparison of
sentence processing theories on naturalistic text. I find that both theories can explain
some of the variance in the eye-movement data, and that they capture different aspects
of sentence processing (Chapter 5).
In Chapter 6, I propose a new theory of sentence processing, which explicitly models
prediction and verification processes, and aims to unify the complementary aspects
of Surprisal and DLT. The proposed theory implements key cognitive concepts such
as incrementality, full connectedness, and memory decay. The underlying grammar
formalism is a strictly incremental version of Tree-adjoining Grammar (TAG), Psycholinguistically
motivated TAG (PLTAG), which is introduced in Chapter 7. I then
describe how the Penn Treebank can be converted into PLTAG format and define an
incremental, fully connected broad-coverage parsing algorithm with associated probability
model for PLTAG. Evaluation of the PLTAG model shows that it achieves the broad coverage required for testing a psycholinguistic theory on naturalistic data. On
the standardized Penn Treebank test set, it approaches the performance of incremental
TAG parsers without prediction (Chapter 8).
Chapter 9 evaluates the psycholinguistic aspects of the proposed theory by testing
it both on a on a selection of established sentence processing phenomena and on the
Dundee eye-tracking corpus. The proposed theory can account for a larger range of
psycholinguistic case studies than previous theories, and is a significant positive predictor
of reading times on broad-coverage text. I show that it can explain a larger
proportion of the variance in reading times than either DLT integration cost or Surprisal
Knowledge Expansion of a Statistical Machine Translation System using Morphological Resources
Translation capability of a Phrase-Based Statistical Machine Translation (PBSMT) system mostly depends on parallel data and phrases that are not present in the training data are not correctly translated. This paper describes a method that efficiently expands the existing knowledge of a PBSMT system without adding more parallel data but using external morphological resources. A set of new phrase associations is added to translation and reordering models; each of them corresponds to a morphological variation of the source/target/both phrases of an existing association. New associations are generated using a string similarity score based on morphosyntactic information. We tested our approach on En-Fr and Fr-En translations and results showed improvements of the performance in terms of automatic scores (BLEU and Meteor) and reduction of out-of-vocabulary (OOV) words. We believe that our knowledge expansion framework is generic and could be used to add different types of information to the model.JRC.G.2-Global security and crisis managemen
Sentence Simplification for Text Processing
A thesis submitted in partial fulfilment of the requirement of the University of Wolverhampton for the degree of Doctor of Philosophy.Propositional density and syntactic complexity are two features of sentences which
affect the ability of humans and machines to process them effectively. In this
thesis, I present a new approach to automatic sentence simplification which processes
sentences containing compound clauses and complex noun phrases (NPs)
and converts them into sequences of simple sentences which contain fewer of these
constituents and have reduced per sentence propositional density and syntactic
complexity.
My overall approach is iterative and relies on both machine learning and handcrafted
rules. It implements a small set of sentence transformation schemes, each
of which takes one sentence containing compound clauses or complex NPs and
converts it one or two simplified sentences containing fewer of these constituents
(Chapter 5). The iterative algorithm applies the schemes repeatedly and is able
to simplify sentences which contain arbitrary numbers of compound clauses and
complex NPs. The transformation schemes rely on automatic detection of these
constituents, which may take a variety of forms in input sentences. In the thesis, I
present two new shallow syntactic analysis methods which facilitate the detection
process.
The first of these identifies various explicit signs of syntactic complexity in
input sentences and classifies them according to their specific syntactic linking and bounding functions. I present the annotated resources used to train and
evaluate this sign tagger (Chapter 2) and the machine learning method used to
implement it (Chapter 3). The second syntactic analysis method exploits the sign
tagger and identifies the spans of compound clauses and complex NPs in input
sentences. In Chapter 4 of the thesis, I describe the development and evaluation
of a machine learning approach performing this task. This chapter also presents
a new annotated dataset supporting this activity.
In the thesis, I present two implementations of my approach to sentence simplification.
One of these exploits handcrafted rule activation patterns to detect
different parts of input sentences which are relevant to the simplification process.
The other implementation uses my machine learning method to identify
compound clauses and complex NPs for this purpose.
Intrinsic evaluation of the two implementations is presented in Chapter 6 together
with a comparison of their performance with several baseline systems. The
evaluation includes comparisons of system output with human-produced simplifications,
automated estimations of the readability of system output, and surveys
of human opinions on the grammaticality, accessibility, and meaning of automatically
produced simplifications.
Chapter 7 presents extrinsic evaluation of the sentence simplification method
exploiting handcrafted rule activation patterns. The extrinsic evaluation involves
three NLP tasks: multidocument summarisation, semantic role labelling, and information
extraction. Finally, in Chapter 8, conclusions are drawn and directions
for future research considered
Wide-coverage statistical parsing with minimalist grammars
Syntactic parsing is the process of automatically assigning a structure to a string
of words, and is arguably a necessary prerequisite for obtaining a detailed and precise
representation of sentence meaning. For many NLP tasks, it is sufficient to use
parsers based on simple context free grammars. However, for tasks in which precision
on certain relatively rare but semantically crucial constructions (such as unbounded
wh-movements for open domain question answering) is important, more expressive
grammatical frameworks still have an important role to play.
One grammatical framework which has been conspicuously absent from journals
and conferences on Natural Language Processing (NLP), despite continuing to dominate
much of theoretical syntax, is Minimalism, the latest incarnation of the Transformational
Grammar (TG) approach to linguistic theory developed very extensively
by Noam Chomsky and many others since the early 1950s. Until now, all parsers
using genuine transformational movement operations have had only narrow coverage
by modern standards, owing to the lack of any wide-coverage TG grammars or treebanks
on which to train statistical models. The received wisdom within NLP is that
TG is too complex and insufficiently formalised to be applied to realistic parsing tasks.
This situation is unfortunate, as it is arguably the most extensively developed syntactic
theory across the greatest number of languages, many of which are otherwise
under-resourced, and yet the vast majority of its insights never find their way into NLP
systems. Conversely, the process of constructing large grammar fragments can have
a salutary impact on the theory itself, forcing choices between competing analyses of
the same construction, and exposing incompatibilities between analyses of different
constructions, along with areas of over- and undergeneration which may otherwise go
unnoticed.
This dissertation builds on research into computational Minimalism pioneered by
Ed Stabler and others since the late 1990s to present the first ever wide-coverage Minimalist
Grammar (MG) parser, along with some promising initial experimental results.
A wide-coverage parser must of course be equipped with a wide-coverage grammar,
and this dissertation will therefore also present the first ever wide-coverage MG, which
has analyses with a high level of cross-linguistic descriptive adequacy for a great many
English constructions, many of which are taken or adapted from proposals in the mainstream
Minimalist literature. The grammar is very deep, in the sense that it describes
many long-range dependencies which even most other expressive wide-coverage grammars
ignore. At the same time, it has also been engineered to be highly constrained,
with continuous computational testing being applied to minimize both under- and over-generation.
Natural language is highly ambiguous, both locally and globally, and even with a
very strong formal grammar, there may still be a great many possible structures for a
given sentence and its substrings. The standard approach to resolving such ambiguity
is to equip the parser with a probability model allowing it to disregard certain unlikely
search paths, thereby increasing both its efficiency and accuracy. The most successful
parsing models are those extracted in a supervised fashion from labelled data in the
form of a corpus of syntactic trees, known as a treebank. Constructing such a treebank
from scratch for a different formalism is extremely time-consuming and expensive,
however, and so the standard approach is to map the trees in an existing treebank into
trees of the target formalism. Minimalist trees are considerably more complex than
those of other formalisms, however, containing many more null heads and movement
operations, making this conversion process far from trivial. This dissertation will describe
a method which has so far been used to convert 56% of the Penn Treebank trees
into MG trees. Although still under development, the resulting MGbank corpus has
already been used to train a statistical A* MG parser, described here, which has an
expected asymptotic time complexity of O(n3); this is much better than even the most
optimistic worst case analysis for the formalism