1,485 research outputs found
A Machine learning approach to POS tagging
We have applied inductive learning of statistical decision trees
and relaxation labelling to the Natural Language Processing (NLP)
task of morphosyntactic disambiguation (Part Of Speech Tagging).
The learning process is supervised and obtains a language
model oriented to resolve POS ambiguities. This model consists
of a set of statistical decision trees expressing distribution of
tags and words in some relevant contexts.
The acquired language models are complete enough to be directly
used as sets of POS disambiguation rules, and include more complex
contextual information than simple collections of n-grams usually
used in statistical taggers.
We have implemented a quite simple and fast tagger that has been
tested and evaluated on the Wall Street Journal (WSJ) corpus with
a remarkable accuracy.
However, better results can be obtained by translating the trees
into rules to feed a flexible relaxation labelling based tagger.
In this direction we describe a tagger which is able to use
information of any kind (n-grams, automatically acquired constraints,
linguistically motivated manually written constraints, etc.), and in
particular to incorporate the machine learned decision trees.
Simultaneously, we address the problem of tagging when only
small training material is available, which is crucial in any process
of constructing, from scratch, an annotated corpus. We show that quite
high accuracy can be achieved with our system in this situation.Postprint (published version
Use of Weighted Finite State Transducers in Part of Speech Tagging
This paper addresses issues in part of speech disambiguation using
finite-state transducers and presents two main contributions to the field. One
of them is the use of finite-state machines for part of speech tagging.
Linguistic and statistical information is represented in terms of weights on
transitions in weighted finite-state transducers. Another contribution is the
successful combination of techniques -- linguistic and statistical -- for word
disambiguation, compounded with the notion of word classes.Comment: uses psfig, ipamac
Methods for Amharic part-of-speech tagging
The paper describes a set of experiments
involving the application of three state-of-
the-art part-of-speech taggers to Ethiopian
Amharic, using three different tagsets.
The taggers showed worse performance
than previously reported results for Eng-
lish, in particular having problems with
unknown words. The best results were
obtained using a Maximum Entropy ap-
proach, while HMM-based and SVM-
based taggers got comparable results
Comparing a statistical and a rule-based tagger for German
In this paper we present the results of comparing a statistical tagger for
German based on decision trees and a rule-based Brill-Tagger for German. We
used the same training corpus (and therefore the same tag-set) to train both
taggers. We then applied the taggers to the same test corpus and compared their
respective behavior and in particular their error rates. Both taggers perform
similarly with an error rate of around 5%. From the detailed error analysis it
can be seen that the rule-based tagger has more problems with unknown words
than the statistical tagger. But the results are opposite for tokens that are
many-ways ambiguous. If the unknown words are fed into the taggers with the
help of an external lexicon (such as the Gertwol system) the error rate of the
rule-based tagger drops to 4.7%, and the respective rate of the statistical
taggers drops to around 3.7%. Combining the taggers by using the output of one
tagger to help the other did not lead to any further improvement.Comment: 8 page
On the Evaluation and Comparison of Taggers: The Effect of Noise in Testing Corpora
This paper addresses the issue of {\sc pos} tagger evaluation. Such
evaluation is usually performed by comparing the tagger output with a reference
test corpus, which is assumed to be error-free. Currently used corpora contain
noise which causes the obtained performance to be a distortion of the real
value. We analyze to what extent this distortion may invalidate the comparison
between taggers or the measure of the improvement given by a new system. The
main conclusion is that a more rigorous testing experimentation
setting/designing is needed to reliably evaluate and compare tagger accuracies.Comment: Appears in proceedings of joint COLING-ACL 1998, Montreal, Canad
- …