This paper introduces some key aspects of machine translation in order to situate the role of the bilingual lexicon in transfer-based systems. It then discusses the data-driven approach to extracting bilingual knowledge automatically from bilingual texts, tracing the processes of alignment at different levels of granularity. The paper concludes with some suggestions for future work.peer-reviewe

2nd Computer Science Annual Workshop (CSAW’04)

Rosner, Michael

OAR@UM

Corpus-Driven Bilingual Lexicon ExtractionMichael RosnerDepartment of Computer Science and AI,University of MaltaAbstract. This paper introduces some key aspects of machine translation in order to situatethe role of the bilingual lexicon in transfer-based systems. It then discusses the data-drivenapproach to extracting bilingual knowledge automatically from bilingual texts, tracing theprocesses of alignment at different levels of granularity. The paper concludes with somesuggestions for future work.1 Machine Translationtransferparsing generationsource languageparse treetarget languageparse treesource language words target language wordsFig. 1.The Machine Translation (MT) problem is almost as old as Computer Science, the first efforts atsolving it dating from the beginning of the 1950s. At that time, early enthusiasm with the novel andapparently limitless capabilities of digital computers led to grandiose claims that the MT problemwould be cracked by purely technological approaches.However, researchers not only failed to appreciate the computational complexity of the problem butalso the role of non-technological factors: syntactic and semantic knowledge, and also subtle culturalconventions that play an important role in distinguishing a good translation from a meaninglessone.By ignoring these factors the systems developed by the early researchers were hopelessly inadequatein terms of both coverage and quality of translation. As a consequence, almost all of the funding forMT dried up in the mid-1960s with a recommendation that more investigation into fundamentallinguistic issues and their computational properties was necessary if MT was to move ahead.This certainly happened. For the next twenty years the emerging field of computational linguisticsconcentrated upon the development of grammar formalisms – special purpose notations createdwith the twin goals of (i) encoding linguistic knowledge and (ii) determining computations.Corpus-Driven Bilingual Lexicon Extraction 79The emphasis on linguistic issues, and on the machinery required to handle them, left its impactupon the proposed architectures for MT systems.Most Machine Translation (MT) systems are transfer based and figure 1 shows the architectureof a typical transfer-based MT system1. The oval items on the left and right represent source andtarget texts, whilst the arrowed lines denote the three different translation phases.– Parsing. A representation (which need not necessarily be a parse tree, as indicated in thefigure) of the surface text is computed, as represented by the left edge of the triangle.– Transfer. The source level representation is transferred to a target representation.– Generation. A surface text in the target langauge is generated from the target level repre-sentation.The essential feature of transfer based systems is that the representation level is not abstractbut linguistic: a representation from source language S still contains elements (e.g. words) fromlanguage S.A great deal of the character of the system depends upon the depth of the representation chosenfor transfer. A shallow representation (i.e. one that is close to the surface text), will be relativelyeasy to compute. From such a representation, it will be relatively easy to generate text. However,precisely because it is shallow, slight variations in the surface form will have to be dealt with bythe transfer component. Conversely, a deeper representation, which abstracts away from much ofthe surface detail, will cause the transfer component to shrink.Different levels of representation yield a spectrum of different solutions to the MT problem. At oneextreme there is Example Based Machine Translation (EBMT) in which there is no representationapart from the text itself. In such systems, the transfer component is essentially lookup (withappropriate indexing) into a database containing pairs of sentences or segment fragments.At the other extreme are systems like Rosetta (Landsbergen [6]), in which the representation isinterlingual (i.e. language free encoding of meaning - see figure 1). Here, the there is no transfercomponent at all, but the analysis and generation phases are substantial.1.1 TransferThe majority of systems fall somewhere in between the two extremes. Systran, for example, whichis the technology underlying Babelfish as used by Alta Vista and Google, uses a low level repre-sentation (see Wheeler [9]) that includes a limited amount of syntactic information.The transfer component involves two somewhat different processes.– Structural transfer deals with the syntactic transformations that are necessary to translate thesyntactic conventions of the source language to that of the target. Figure 1.1 shows a simpleexample that reorders adjectives and nouns (cf. English and Maltese).– Lexical transfer concerns the actual translations between words. The system component thatdeals with this kind of equivalence is the bilingual lexicon, as discussed in the next section.1 Figure due to Jurafsky and Martin [5]80 Michael Rosnertransferparsing generationsource languageparse treetarget languageparse treesource language words target language wordsInterlinguainterpretation generationFig. 2.noun phraseadjective nounnoun phraseadjectivenounFig. 3.1.2 The Bilingual LexiconA key element of any translation system is a bilingual lexicon, which, as its name suggests, isessentially a lookup table which encodes mappings between words in the two languages under con-sideration. The construction of such a lexicon is both non-trivial and highly labour-intensive. Thenon-triviality arises from several sources. First is the inherently difficult nature of the equivalencerelation between words in different languages: for a given word there may be zero to N differentpossible translations - and where N is greater than 1, the choice may depend upon complex aspectsof the context. Second is that the number of words is large: a typical dictionary contains c. 100,000entries. Even if we ignore the problem of multiple-word expressions and morphological complexity,the creation of such a lexicon is a prodigious task: if each entry takes 15 mins to create (probablyan underestimate) this corresponds to 12.5 man-years of work.In short, then, the creation of a bilingual dictionary constitutes a knowledge acquisition bottleneck,and computational linguists have turned to data-driven rather than handcrafted approaches tolexicon construction.2 The Data Driven ApproachProfessional translators know that the richest source of translation knowledge are texts or textfragments that have already been translated into another language. The question is whether thisintuition can be harnessed by an automatic procedure for extracting translation knowledge ingeneral, and a bilingual lexicon in particular, from particular instances of translated texts.Corpus-Driven Bilingual Lexicon Extraction 81In order to develop this idea, we will regard a text S as a set of segments. Segments are simplylinguistic entities of a certain type, where the types vary according to the level of analysis. Thus ata given level of analysis, a segment might be a sequence of words, a sequence of paragraphs, or anentire chapter. Furthermore, other relations might be defined over a set of such segments. Withina sentence text, for example, sequence-of-word segments will be ordered. However, for the presentwe will not be concerned with either the internal character of segments nor with the relations thatmight exist over them. The important intuition is that the text is a set.3 AlignmentWithin the framework under discussion, bilingual knowledge is derived from correspondences be-tween a text S and its translation T, that is, between the respective segments in S = {s1, s2, . . . , sn}and T = {t1, t2, . . . , tm}. These correspondences are expressed using the notion of alignment, de-fined by Isabelle and Simard [4] as a subset of the Cartesian product S × T .To give an example, supposeS = {s1, s2, s3, s4, s5}andT = {t1, t2, t3, t4, t5}A particular alignment might beA = {(s1, t1), (s2, t2), (s2, t3), (s3, t4), (s4, t5), (s5, t5)}This associates the segment s1 with segment t1; the segment s2 with segments t2 and t3; thesegment s3 to segment t4; and the segments s4 and s5 to the same segment t5.A bilingual text, or “bitext” is a triple (A,S,T) where A is an alignment, S is a text, and T is itstranslation.Input to an alignment system is normally the bitext ({(s,t)},{s},{t}), i.e. comprising a single, largetext segment, its translation , and the alignment that is the one-to-one mapping between the two.In order to derive bilingual lexical information from this initial input, a bitext at the next levelmust be constructed by– identifying the subsegments of S and T– computing the best alignment of the subsegments of S and T.This process is repeated until subsegmentation reaches the word level where, potentially everysubset of words in the source text can be aligned with every subset of words in the target text.At this point we inspect the best alignment and count occurrences of similar pairs. The highestranking pairs are frequently occurring n-gram-to-n-gram translations and therefore good candidatesfor the bilingual lexicon.Of course, this is a deliberate simplification of the process. When, at a given level, the number ofsubsegments is large, the cost of computing the possible alignments of all possible subsegments isprohibitively high so in practice certain heuristic assumptions must be incorporated to reduce thiscost, e.g.82 Michael Rosner– Subsegments are defined in such a way that that the number of them at a given level isreasonable. This is achieved by staging the alignment process to deal, in order, with alignmentsof sections, paragraphs, sentences, and finally words.– Limits are imposed on the set of candidate alignments that will actually be considered. Whenperforming subsentence alignment, for example, one can put an upper bound on their length.Hence, most work in this area in fact deals with single words. We shall see other examplesbelow.3.1 Paragraph AlignmentTypically, segmentation of paragraphs is carried out in two stages: anchor alignment and paragraphalignment, as described originally by Brown et. al. [2]. A slightly simplified procedure was adoptedfor the Maltese/English work by Bajada [1].The two texts are inspected and a set of anchor point types manually identified. An anchor point isa point that is easily recognised in both source and target texts such as a chapter heading, title, orother mark that is characteristic of the text type under investigation. The identification of anchorpoints divides the two texts into sections containing one or more paragraphs, as shown in figure 4.A table of anchor point types together with their respective translations is compiled and stored inmemory (to be used for computing alignment costs)..--- .S4 sectionS1 sectionsectionsectionS2S3anchor pointanchor pointanchor pointFig. 4.For the most part, corresponding anchor points are present in both source and target texts. How-ever, there are typically omissions of one or other anchor and/or minor departures from the exactanchor translations as stored in the table. Hence, the automatic alignment of anchors is not entirelystraightforward.Both Brown et. al. and Bajada adopted the technique of assigning a cost to the alignment of aproposed source/target anchor pairing. If the proposed pair is in the table, the cost is zero. If oneelement of the pair is an omission, a fixed cost is assigned. Otherwise, rules can be supplied fordealing with concrete cases. In the case of Bajada, a dynamic programming algorithm to find theleast cost alignment is based on a limited pattern of possibilities for the next anchor pair.Corpus-Driven Bilingual Lexicon Extraction 83Specifically, two lists of anchor points are created, one for the source and one for the target. Theordering within these list preserves the ordering of anchor points in the texts. The algorithm iteratesthrough each source anchor point for which the first N target anchor points are compared, whereN is a small integer, and the one with the least cost assigned. This is an example of limiting the setof candidate alignments that will actually be considered, as mentioned above. In the work carriedout by Bajada, a value of 3 was chosen for N, since it was established empirically that for the textsunder consideration, the anchor points were never out of alignment by more than 3.The quality of the results obtained depends largely on the reliability of the anchor points identifiedby hand as indicators of “section-ness”. Although these can be very reliable in formalised textsuch as legal documents, the problem is harder in the case of, for example, government circularsor communications from the local Water Services. Simard et. al [8] have proposed using linguisticcognates as the basis for recognising potential anchor points automatically.Anchor points provide a rough alignment of source and target sections each containing severalunaligned paragraphs.The next stage aligns the paragraphs within sections. For each aligned anchor pair, the algorithmfirst retrieves and then counts the number of source and target paragraphs they enclose. Let N andM be the respective counts.Following the strategy of limiting the set of possible alignments, Bajada allows the following align-ment options, for n,m > 1:0 : 0, 1 : 1, 1 : 0, 0 : 1, 1 : n, n : 1, n : n, n : m(n < m), n : m(m > n), 1 : 0, 0 : 1where X:Y means that X source paragraphs are aligned with Y target paragraphs. Clearly, thechoice is fully determined for all but the last two cases, for which all possible combinations areenumerated and ranked by considering the length, in characters, of individual paragraphs or con-catenations of paragraphs. This strategy proved to be tractable because the number of paragraphsunder consideration was always small for the texts being considered.The underlying assumption is that the length ratio of correctly aligned paragraphs is approximatelyconstant for a given language pair. Hence the cost of a given alignment is proportional to the sum,over all source/target pairs in the alignment, of the difference in character length between eachsource/target pair, and the least-cost alignment is that which minimises the sum.3.2 Sentence AlignmentLike paragraph alignment, the sentence alignment algorithms proposed by Brown et. al. [2], andGale and Church [3], are based on the length of sentences in characters. The latter was reimple-mented by Bajada [1]. Like paragraph alignment, both algorithms assume that longer sentencestend to be translated by longer sentences and vice versa. The operation of the algorithm is sum-marised by Bajada as follows:“The algorithm calculates the cost of aligning sentences by assigning a probabilisticscore to each possible pair of sentences which make up an alignment. The score is assignedaccording to the ratio of the lengths of the two sentences and on the variance of thisratio for all sentences. Dynamic programming is then used to find the maximum likelihoodalignment of the sentences.”84 Michael Rosner3.3 Word AlignmentIn a classic paper that outlines the basic tenets of statistical machine translation, Brown et. al. [7]develop a model in which for estimating Pr(f|e), the probability that f, a target (French) sentence,is the translation of e, a source (English) sentence. This is of course defined in terms of alignmentsexpressed as sets of what they term connections.A connection, written conf,ej,i , states that position j in f is connected to position i in e. The modelis directional, so that each position in f is connected to exactly one position in e, including thespecial “null” position 0 (where it is not connected to any word). Consequently, the number ofconnections in a given alignment is equal to the length of f.The translation probability for a pair of sentences is expressed asPr(f |e) = Σa∈APr(f, a|e)where A is the set of all possible alignments for the sentences f and e. In other words, each possiblealignment contributes its share of probability additively. The question is, how to estimate Pr(f, a|e).Brown et. al. propose 5 different models.The first model assumes that Pr(f, a|e) depends purely on the translation probabilities t(φ|)between the word pairs (φ, ) contained in the connections of a, and is obtained by multiplyingthese probabilities together. The individual values for t(φ|) are estimated using a training setcomprising a bitext that has been aligned at sentence level.The second model improves upon the first by taking into account the observation that, undertranslation, and for certain language pairs at least, words tend to retain their relative position inthe sentence: words that are near the beginning of the source sentence tend to appear near thebeginning of the target sentence.3.4 ConclusionBajada’s FYP report [1] implemented a complete framework for the extraction of bilingual lexicalequivalences that included all of the other alignment phases described above, including the twoversions of word alignment mentioned above. Using training and test material based on an abridgedversion of the Malta-EU accession treaty, results for section, paragraph and sentence alignment wereencouraging, with precision and recall above 90% over the test material.The results for word alignment were somewhat less satisfactory, with precision and recall figurescloser to 50%. It was shown that the second model for word alignment described above was animprovement on the first.Amongst the strategies under consideration for improving the performance of the system at wordlevel are:– Investigation of larger datasets in order to determine the rate of improvement for a givenincrease in size.– Use of a lemmatiser to reduce the vocabulary size and hence the frequency of individual con-nections between source and target words (this will be of particular interest on the Malteseside).– Investigation of possible equivalences between frequent n-grams of words for given n.Corpus-Driven Bilingual Lexicon Extraction 85References1. J. Bajada. Investigation of translation equivalences from parallel texts. University of Malta, BSc. FinalReport, 2004.2. P. Brown, J. Lai, and R. Mercer. Aligning sentences in parallel corpora. In 29th Meeting of theAssociation for Computational Linguistics, pages 169–176, 1991.3. W. Gale and K. Church. A program for aligning sentences in bilingual corpora. In 29th Meeting of theAssociation for Computational Linguistics, pages 177–184, 1991.4. Pierre Isabelle and Michel Simard. Propositions pour la reprsentation et lvaluation des alignements etdes textes parallles. Rapport technique du CITI., 1996.5. D. Jurafsky and J. Martin. Speech and Language Processing. Prentice-Hall, 2000.6. J. Landsbergen. Isomorphic grammars and their use in the rosetta translation system. In M. King,editor, Machine Translation Today. Edinburgh University Press, 1997.7. Brown P., J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, and R. Mercer. A statistical approachto language translation. Computational Linguistics, 16.2:79–85, 1990.8. Michel Simard, George F. Foster, and Pierre Isabelle. Using cognates to align sentences in bilingualcorpora. In Proceedings of the 1993 conference of the Centre for Advanced Studies on Collaborativeresearch, pages 1071–1082. IBM Press, 1993.9. P. Wheeler. Systran. In M. King, editor, Machine Translation Today, pages 192–208. EdinburghUniversity Press, 1997.

Corpus-driven bilingual lexicon extraction

https://www.um.edu.mt/library/oar//bitstream/123456789/22446/1/Proceedings%20of%20CSAW%e2%80%9904%20-%20A10.pdf

Corpus-driven bilingual lexicon extraction

Abstract

Similar works

Full text

Available Versions

OAR@UM