23 research outputs found
Bidirectional dependency parsing trained on the Turin University Treebank
In this paper, we describe the application of a bidirectional dependency parser trained on the Turin University Treebank
Dependency-based Convolutional Neural Networks for Sentence Embedding
In sentence modeling and classification, convolutional neural network
approaches have recently achieved state-of-the-art results, but all such
efforts process word vectors sequentially and neglect long-distance
dependencies. To exploit both deep learning and linguistic structures, we
propose a tree-based convolutional neural network model which exploit various
long-distance relationships between words. Our model improves the sequential
baselines on all three sentiment and question classification tasks, and
achieves the highest published accuracy on TREC.Comment: this paper has been accepted by ACL 201
Null Element Restoration
Understanding the syntactic structure of a sentence is a necessary preliminary to understanding its semantics and therefore for many practical applications. The field of natural language processing has achieved a high degree of accuracy in parsing, at least in English. However, the syntactic structures produced by the most commonly used parsers are less detailed than those structures found in the treebanks the parsers were trained on. In particular, these parsers typically lack the null elements used to indicate wh-movement, control, and other phenomena.
This thesis presents a system for inserting these null elements into parse trees in English. It then examines the problem in Arabic, which motivates a second, joint- inference system which has improved performance on English as well. Finally, it examines the application of information derived from the Google Web 1T corpus as a way of reducing certain data sparsity issues related to wh-movement
Lexicalized semi-incremental dependency parsing
Even leaving aside concerns of cognitive plausibility,
incremental parsing is appealing for applications such
as speech recognition and machine translation because
it could allow for incorporating syntactic features into
the decoding process without blowing up the search
space. Yet, incremental parsing is often associated
with greedy parsing decisions and intolerable loss of
accuracy. Would the use of lexicalized grammars provide
a new perspective on incremental parsing? In this paper we explore incremental left-to-right dependency parsing using a lexicalized grammatical formalism that works with lexical categories (supertags) and a small set of combinatory operators. A strictly incremental parser would conduct only a single pass over the input, use no lookahead and make only local decisions at every word. We show that such a parser suffers heavy loss of accuracy. Instead, we explore
the utility of a two-pass approach that incrementally
builds a dependency structure by first assigning a supertag
to every input word and then selecting an incremental
operator that allows assembling every supertag with the dependency structure built so-far to its left. We instantiate this idea in different models that allow
a trade-off between aspects of full incrementality
and performance, and explore the differences between
these models empirically. Our exploration shows that
a semi-incremental (two-pass), linear-time parser that
employs fixed and limited look-ahead exhibits an appealing
balance between the efficiency advantages of incrementality and the achieved accuracy. Surprisingly, taking local or global decisions matters very little for the accuracy of this linear-time parser. Such a parser fits seemlessly with the currently dominant finite-state decoders for machine translation
Broad-coverage model of prediction in human sentence processing
The aim of this thesis is to design and implement a cognitively plausible theory
of sentence processing which incorporates a mechanism for modeling a prediction
and verification process in human language understanding, and to evaluate the validity
of this model on specific psycholinguistic phenomena as well as on broad-coverage,
naturally occurring text.
Modeling prediction is a timely and relevant contribution to the field because recent
experimental evidence suggests that humans predict upcoming structure or lexemes
during sentence processing. However, none of the current sentence processing theories
capture prediction explicitly. This thesis proposes a novel model of incremental
sentence processing that offers an explicit prediction and verification mechanism.
In evaluating the proposed model, this thesis also makes a methodological contribution.
The design and evaluation of current sentence processing theories are usually
based exclusively on experimental results from individual psycholinguistic experiments
on specific linguistic structures. However, a theory of language processing in
humans should not only work in an experimentally designed environment, but should
also have explanatory power for naturally occurring language.
This thesis first shows that the Dundee corpus, an eye-tracking corpus of newspaper
text, constitutes a valuable additional resource for testing sentence processing theories.
I demonstrate that a benchmark processing effect (the subject/object relative clause
asymmetry) can be detected in this data set (Chapter 4). I then evaluate two existing
theories of sentence processing, Surprisal and Dependency Locality Theory (DLT),
on the full Dundee corpus. This constitutes the first broad-coverage comparison of
sentence processing theories on naturalistic text. I find that both theories can explain
some of the variance in the eye-movement data, and that they capture different aspects
of sentence processing (Chapter 5).
In Chapter 6, I propose a new theory of sentence processing, which explicitly models
prediction and verification processes, and aims to unify the complementary aspects
of Surprisal and DLT. The proposed theory implements key cognitive concepts such
as incrementality, full connectedness, and memory decay. The underlying grammar
formalism is a strictly incremental version of Tree-adjoining Grammar (TAG), Psycholinguistically
motivated TAG (PLTAG), which is introduced in Chapter 7. I then
describe how the Penn Treebank can be converted into PLTAG format and define an
incremental, fully connected broad-coverage parsing algorithm with associated probability
model for PLTAG. Evaluation of the PLTAG model shows that it achieves the broad coverage required for testing a psycholinguistic theory on naturalistic data. On
the standardized Penn Treebank test set, it approaches the performance of incremental
TAG parsers without prediction (Chapter 8).
Chapter 9 evaluates the psycholinguistic aspects of the proposed theory by testing
it both on a on a selection of established sentence processing phenomena and on the
Dundee eye-tracking corpus. The proposed theory can account for a larger range of
psycholinguistic case studies than previous theories, and is a significant positive predictor
of reading times on broad-coverage text. I show that it can explain a larger
proportion of the variance in reading times than either DLT integration cost or Surprisal
MICA: A Probabilistic Dependency Parser Based on Tree Insertion Grammars
International audienceMICA is a dependency parser which returns deep dependency representations, is fast, has state-of-the-art performance, and is freely available
A psycholinguistically motivated version of TAG
We propose a psycholinguistically moti-vated version of TAG which is designed to model key properties of human sentence processing, viz., incrementality, connect-edness, and prediction. We use findings from human experiments to motivate an in-cremental grammar formalism that makes it possible to build fully connected struc-tures on a word-by-word basis. A key idea of the approach is to explicitly model the prediction of upcoming material and the subsequent verification and integration pro-cesses. We also propose a linking theory that links the predictions of our formalism to experimental data such as reading times, and illustrate how it can capture psycholin-guistic results on the processing of either... or structures and relative clauses.
Syntax und Valenz: Zur Modellierung kohärenter und elliptischer Strukturen mit Baumadjunktionsgrammatiken
Diese Arbeit untersucht das Verhältnis zwischen Syntaxmodell und lexikalischen Valenzeigenschaften anhand der Familie der Baumadjunktionsgrammatiken (TAG) und anhand der Phänomenbereiche Kohärenz und Ellipse. Wie die meisten prominenten Syntaxmodelle betreibt TAG eine Amalgamierung von Syntax und Valenz, die oft zu Realisierungsidealisierungen führt. Es wird jedoch gezeigt,
dass TAG dabei gewisse Realisierungsidealisierungen vermeidet und Diskontinuität bei Kohärenz direkt repräsentieren kann;
dass TAG trotzdem und trotz der im Vergleich zu GB, LFG und HPSG wesentlich eingeschränkten Ausdrucksstärke zu einer linguistisch sinnvollen Analyse kohärenter Konstruktionen herangezogen werden kann;
dass der TAG-Ableitungsbaum für die indirekte Gapping-Modellierung eine ausreichend informative Bezugsgröße darstellt.
Für die direkte Repräsentation von Gapping-Strukturen wird schließlich ein baumbasiertes Syntaxmodell, STUG, vorgeschlagen, in dem Syntax und Valenz getrennt, aber verlinkt sind.
German law requires we state the prices in Germany for this publication. The hardcover price is 35.00 EUR; the softcover price is 25.00 EUR