Simões, Alberto

English

Repositório Comum

EBMT: Example Based Machine TranslationAlberto Manuel Branda˜o Simo˜es — ambs@di.uminho.ptExample-Based Machine Translation (EBMT) is a statistical based method for Machine Translation . The idea behind thisapproach to MT is that if we take all the translations already done in the world, probably we will need to translate just a few more sentences, or justtranslate some words.So, EBMT uses a database of translations , called examples (this database is in many cases just parallel corpora, but its complexity can raiseto more complex data structures like syntactic trees), where it will search for translations similar to the one being performed. After extracting theseexamples, they will be merged in a full and, hopefully, correct translation[7].Although based on a simple concept, this translation method is one of the main research areas in MT. New techniques are appearing in the storingmethods with examples generalization, matching using word morphological information and adapting the examples using rules. In this documentwe will try to discuss each one of these steps of MT and how previous work done with word-alignment of parallel corpora can be useful for this task.We propose to develop prototypes for some pieces of this tool, in special regarding Portuguese↔ English translation, and research what techniquesfor each one of the four components can be more effective for the Portuguese language . Finally, an evaluation process will be defined so we candiscuss the results of applying the different techniques during EBMT.Examples DatabaseData TypeIt is important to specify a correct way to store examples. Also, the definition of exampleis not trivial.tmdb = transMemory : tm ×transPattern : tp ×dictionary : dictm = alignUnitm↔ alignUnittp = patternm↔ patterndic = word ⇀ wordInfowordInfo = trans : word ⇀ real ×occur : intThe traditional definition of a translation memory database (tmdb) include just pairs ofsentences to be used directly (in the called Memory Based Machine Translation). In thisdefinition we call them alignUnits because they do not need to be full clauses.Taking sentences together it is possible to remove common parts and create place-holders.To these generalized align units we call patterns[2].Finally, if we divide more and more the alignUnits, we get pairs of words. The trans-lation database should include a dictionary[8] (in this case, a probabilistic translationdictionary[6]) for word-to-word translations.Generalization of ExamplesGiven the scarcity of parallel corpora, the examples should be generalized and processedin order to generate more examples.The first simple way is to detect translation units. This can be done using chunkers,syntactic sentence information and multi-term or compound verbs information.Splitting sentences into words is similar to the corpora word alignments. A tool likeNATools[4] can be used to extract this alignment which results in a probabilistic transla-tion dictionary.Patterns generalize examples so we can use them in sentences with similar constructs andwords, but that are not exactly the same. Generalizing techniques[2] include:• word chunking: detection of words used in similar contexts;• entity name recognition;• part-of-speech tagging, to create word classes;• generalization removing different portions of similar sentencesMore than just storing examples, it is also important to find ways to create them. Al-though there is freely available parallel corpora, sometimes they do not exist in the re-quired quantities. One way to solve that problem is using techniques to extract examplesfrom the Internet [1].Splitting the SourceIt is very improbable that we find the full sentence thesystem is trying to translate as an example. So, thesystem should split the sentence in smaller segments.split : sentence −→ segment?split(s)def=...The way sentences should be split is not trivial. We haveto split the sentence in sequences that exist in the ex-amples database. Also, these extract do not need to befull phrases as linguists see them: just simple word se-quences that pulled together form the original sentence.This splitting can be done in using different techniques:• use chunkers to create word segments;• use syntactic sentence information — using anno-tated corpora (not necessarily parallel one), likeCETEMPu´blico[3].• multi-term and compound verbs detection — spe-cially if we use a similar technique to create exam-ples.This process should be iterative so we can match differ-ent sequences of words against the examples and use theones that give better results. It is specially importantbecause some methods to split the sentences can divideconstituents that should be translated together.Matching FragmentsWith the sentence segments, we need to find their trans-lations on the examples. For that, we will match thosesegments against the examples. This process should befast, but versatile. The match should be done in a fuzzylike approach.match : tmdb× segment −→ segmentmatch(db, s)def=...For each sequence of words being matched we mustcheck how much probable that translation is. For that,the probabilistic dictionary can be used. This way, wecan return a value to the splitter module which can trywith another word sequences (bigger or smaller) whichcan give better results.Also, the sequence of words can match a pattern. Inthat case, the place holder portion should be matchedagain against the examples database.The matching step can also use some semantic knowl-edge. Word-sense disambiguation both while matchingsentences (matching only if their sense is the same) andchoosing a translation from the dictionary (choosing atranslation with the same sense) can be very helpful.Recombining TranslationFragmentsAfter the examples segments extraction, they need to berecombined. Unfortunately, to recombine the segmenttranslations is not so simple as their concatenates. Infact, some phenomena like gender and number issues aswell as words order should be analyzed.recombine : segment? −→ sentencerecombine(s)def=...Normally this step is performed using a set of rules to-gether with a morphological analyzer[5]. The rules canbe of different complexity accordingly with the problemit tries to solve.Some of these rules can be learned from existing paral-lel corpora. Some other rules (like gender and numberconcordance) can be written in a general way using mor-phological analyzers.translate : sentence× tmdb −→ sentencetranslate(s, db)def=let l1 = split(s)l2 = <match(db, x) | x ∈ l1>in recombine(l2)[1] Jose´ Joa˜o Almeida, Alberto Manuel Simo˜es, and Jose´ Alves Castro. Grabbing parallel corpora from the web. In Sociedade Espan˜ola para el Procesamiento del Lenguaje Natural, 29, pages 13–20, Sep. 2002.[2] Ralf D. Brown. Transfer-rule induction for example-based translation. In Michael Carl and Andy Way, editors, Workshop on Example-Based Machine Translation, pages 1–11, September 2001.[3] Paulo Alexandre Rocha and Diana Santos. Cetempu´blico: Um corpus de grandes dimenso˜es de linguagem jornal´ıstica portuguesa. In Maria das Grac¸as Volpe Nunes, editor, V Encontro para o processamento computacionalda l´ıngua portuguesa escrita e falada (PROPOR 2000), pages 131–140, Atibaia, Sa˜o Paulo, November 2000.[4] Alberto M. Simo˜es and J. Joa˜o Almeida. Natools – a statistical word aligner workbench. SEPLN, Sep. 2003.[5] Alberto Manuel Simo˜es and Jose´ Joa˜o Almeida. jspell.pm – um mo´dulo de ana´lise morfolo´gica para uso em processamento de linguagem natural. In Actas da Associac¸a˜o Portuguesa de Lingu´ıstica, pages 485–495, 2001.[6] Alberto Manuel Branda˜o Simo˜es. Parallel corpora word alignment and applications. Master’s thesis, Escola de Engenharia - Universidade do Minho, 2004.[7] Harold Somers. Review article: Example based machine translation. Machine Translation, 14(2):113–157, 1999.[8] Davide Turcato and Fred Popowich. What is example-based machine translation? In Michael Carl and Andy Way, editors, Workshop on Example-Based Machine Translation, pages 43–48, September 2001.

EBMT: Example Based Machine Translation

https://comum.rcaap.pt/bitstream/10400.26/89/1/SimoesSDUMinho2005Poster.pdf

EBMT: Example Based Machine Translation

Abstract

Similar works

Full text

Available Versions

Repositório Comum