2 research outputs found
Language independent text correction using finite state automata
Many natural language applications, like machine translation and information extraction, are required to operate on text with spelling errors. Those spelling mistakes have to be corrected automatically to avoid deteriorating the performance of such applications. In this work, we introduce a novel approach for automatic correction of spelling mistakes by deploying finite state automata to propose candidates corrections within a specified edit distance from the misspelled word. After choosing candidate corrections, a language model is used to assign scores the candidate corrections and choose best correction in the given context. The proposed approach is language independent and requires only a dictionary and text data for building a language model. The approach have been tested on both Arabic and English text and achieved accuracy of 89%.
Correction of Noisy Sentences using a Monolingual Corpus
Correction of Noisy Natural Language Text is an important and well studied
problem in Natural Language Processing. It has a number of applications in
domains like Statistical Machine Translation, Second Language Learning and
Natural Language Generation. In this work, we consider some statistical
techniques for Text Correction. We define the classes of errors commonly found
in text and describe algorithms to correct them. The data has been taken from a
poorly trained Machine Translation system. The algorithms use only a language
model in the target language in order to correct the sentences. We use phrase
based correction methods in both the algorithms. The phrases are replaced and
combined to give us the final corrected sentence. We also present the methods
to model different kinds of errors, in addition to results of the working of
the algorithms on the test set. We show that one of the approaches fail to
achieve the desired goal, whereas the other succeeds well. In the end, we
analyze the possible reasons for such a trend in performance.Comment: 67 pages, 2 figures, 4 tables, 2 algorithm