3,447 research outputs found
Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection
Grammatical error correction, like other machine learning tasks, greatly
benefits from large quantities of high quality training data, which is
typically expensive to produce. While writing a program to automatically
generate realistic grammatical errors would be difficult, one could learn the
distribution of naturallyoccurring errors and attempt to introduce them into
other datasets. Initial work on inducing errors in this way using statistical
machine translation has shown promise; we investigate cheaply constructing
synthetic samples, given a small corpus of human-annotated data, using an
off-the-rack attentive sequence-to-sequence model and a straight-forward
post-processing procedure. Our approach yields error-filled artificial data
that helps a vanilla bi-directional LSTM to outperform the previous state of
the art at grammatical error detection, and a previously introduced model to
gain further improvements of over 5% score. When attempting to
determine if a given sentence is synthetic, a human annotator at best achieves
39.39 score, indicating that our model generates mostly human-like
instances.Comment: Accepted as a short paper at EMNLP 201
On the Similarities Between Native, Non-native and Translated Texts
We present a computational analysis of three language varieties: native,
advanced non-native, and translation. Our goal is to investigate the
similarities and differences between non-native language productions and
translations, contrasting both with native language. Using a collection of
computational methods we establish three main results: (1) the three types of
texts are easily distinguishable; (2) non-native language and translations are
closer to each other than each of them is to native language; and (3) some of
these characteristics depend on the source or native language, while others do
not, reflecting, perhaps, unified principles that similarly affect translations
and non-native language.Comment: ACL2016, 12 page
Detecting grammatical errors with treebank-induced, probabilistic parsers
Today's grammar checkers often use hand-crafted rule systems that define acceptable language. The development of such rule systems is labour-intensive and has to be repeated for each language. At the same time, grammars automatically induced from syntactically annotated corpora (treebanks) are successfully employed in other applications, for example text understanding and machine translation. At first glance, treebank-induced grammars seem to be unsuitable for grammar checking as they massively over-generate and fail to reject ungrammatical input due to their high robustness. We present three new methods for judging the grammaticality of a sentence with probabilistic, treebank-induced grammars, demonstrating that such grammars can be successfully applied to automatically judge the grammaticality of an input string. Our best-performing method exploits the differences between parse results for grammars trained on grammatical and ungrammatical treebanks. The second approach builds an estimator of the probability of the most likely parse using grammatical training data that has previously been parsed and annotated with parse probabilities. If the estimated probability of an input sentence (whose grammaticality is to be judged by the system) is higher by a certain amount than the actual parse probability, the sentence is flagged as ungrammatical. The third approach extracts discriminative parse tree fragments in the form of CFG rules from parsed grammatical and ungrammatical corpora and trains a binary classifier to distinguish grammatical from ungrammatical sentences. The three approaches are evaluated on a large test set of grammatical and ungrammatical sentences. The ungrammatical test set is generated automatically by inserting common grammatical errors into the British National Corpus. The results are compared to two traditional approaches, one that uses a hand-crafted, discriminative grammar, the XLE ParGram English LFG, and one based on part-of-speech n-grams. In addition, the baseline methods and the new methods are combined in a machine learning-based framework, yielding further improvements
Recommended from our members
Auxiliary Objectives for Neural Error Detection Models
We investigate the utility of different auxiliary
objectives and training strategies
within a neural sequence labeling approach
to error detection in learner writing.
Auxiliary costs provide the model
with additional linguistic information, allowing
it to learn general-purpose compositional
features that can then be exploited
for other objectives. Our experiments
show that a joint learning approach
trained with parallel labels on in-domain
data improves performance over the previous
best error detection system. While
the resulting model has the same number
of parameters, the additional objectives allow
it to be optimised more efficiently and
achieve better performance
Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task
Previously, neural methods in grammatical error correction (GEC) did not
reach state-of-the-art results compared to phrase-based statistical machine
translation (SMT) baselines. We demonstrate parallels between neural GEC and
low-resource neural MT and successfully adapt several methods from low-resource
MT to neural GEC. We further establish guidelines for trustable results in
neural GEC and propose a set of model-independent methods for neural GEC that
can be easily applied in most GEC settings. Proposed methods include adding
source-side noise, domain-adaptation techniques, a GEC-specific
training-objective, transfer learning with monolingual data, and ensembling of
independently trained GEC models and language models. The combined effects of
these methods result in better than state-of-the-art neural GEC models that
outperform previously best neural GEC systems by more than 10% M on the
CoNLL-2014 benchmark and 5.9% on the JFLEG test set. Non-neural
state-of-the-art systems are outperformed by more than 2% on the CoNLL-2014
benchmark and by 4% on JFLEG.Comment: Accepted for oral presentation in long paper research track at NAACL
201
Monolingual Sentence Rewriting as Machine Translation: Generation and Evaluation
In this thesis, we investigate approaches to paraphrasing entire sentences within the constraints of a given task, which we call monolingual sentence rewriting. We introduce a unified framework for monolingual sentence rewriting, and apply it to three representative tasks: sentence compression, text simplification, and grammatical error correction. We also perform a detailed analysis of the evaluation methodologies for each task, identify bias in common evaluation techniques, and propose more reliable practices.
Monolingual rewriting can be thought of as translating between two types of English (such as from complex to simple), and therefore our approach is inspired by statistical machine translation. In machine translation, a large quantity of parallel data is necessary to model the transformations from input to output text. Parallel bilingual data naturally occurs between common language pairs (such as English and French), but for monolingual sentence rewriting, there is little existing parallel data and annotation is costly. We modify the statistical machine translation pipeline to harness monolingual resources and insights into task constraints in order to drastically diminish the amount of annotated data necessary to train a robust system. Our method generates more meaning-preserving and grammatical sentences than earlier approaches and requires less task-specific data.
Once candidate sentences are generated, it is crucial to have reliable evaluation methods. Sentential paraphrases must fulfill a variety of requirements: preserve the meaning of the original sentence, be grammatical, and meet any stylistic or task-specific constraints. We analyze common evaluation practices and propose better methods that more accurately measure the quality of output. Often overlooked, robust automatic evaluation methodology is necessary for improving systems, and this work presents new metrics and outlines important considerations for reliably measuring the quality of the generated text
Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection
Grammatical error correction, like other machine
learning tasks, greatly benefits from
large quantities of high quality training data,
which is typically expensive to produce. While
writing a program to automatically generate
realistic grammatical errors would be difficult,
one could learn the distribution of naturallyoccurring
errors and attempt to introduce them
into other datasets. Initial work on inducing
errors in this way using statistical machine
translation has shown promise; we investigate
cheaply constructing synthetic samples, given
a small corpus of human-annotated data, using
an off-the-rack attentive sequence-to-sequence
model and a straight-forward post-processing
procedure. Our approach yields error-filled artificial
data that helps a vanilla bi-directional
LSTM to outperform the previous state of the
art at grammatical error detection, and a previously
introduced model to gain further improvements
of over 5% F0.5 score. When attempting
to determine if a given sentence is
synthetic, a human annotator at best achieves
39.39
F1 score, indicating that our model generates
mostly human-like instances
- …