We focus on improving the translation of the English pronoun it and English reflexive pronouns in an English-Czech syntax-based machine translation framework. Our evaluation both from intrinsic and extrinsic perspective shows that adding specialized syntactic and coreference-related features leads to an improvement in trans-
lation quality

Nedoluzhko, Anna

Novák, Michal

Žabokrtský, Zdeněk

Biblio at Institute of Formal and Applied Linguistics

International Joint Conference on Natural Language Processing, pages 1037–1041,Nagoya, Japan, 14-18 October 2013.Two Case Studies on Translating Pronouns in a Deep Syntax FrameworkMichal Nova´k, Zdeneˇk Zˇabokrtsky´ and Anna NedoluzhkoCharles University in Prague, Faculty of Mathematics and PhysicsInstitute of Formal and Applied LinguisticsMalostranske´ na´meˇstı´ 25, CZ-11800{mnovak,zabokrtsky,nedoluzko}@ufal.mff.cuni.czAbstractWe focus on improving the translation ofthe English pronoun it and English reflex-ive pronouns in an English-Czech syntax-based machine translation framework.Our evaluation both from intrinsic and ex-trinsic perspective shows that adding spe-cialized syntactic and coreference-relatedfeatures leads to an improvement in trans-lation quality.1 IntroductionMachine Translation (MT) is an extremely broadtask and can be decomposed along various di-rections. One of them lies in using specializedtranslation models (TMs) for certain types of lan-guage expressions. For instance, different typesof named entities often receive specialized treat-ment in real translation systems. This paper dealswith introducing specialized TMs for two types ofpronouns: the pronoun it and reflexive pronouns.The models are integrated into an English-Czechsyntax-based MT framework.Several works have previously focused on trans-lating pronouns. The linguistic study of Morin(2009) investigated the translation of pronouns,proper names and kinship terms from Indonesianinto English. Onderkova´ (2010) has conducted acorpus-based research on possesive pronouns inCzech and English, focusing especially on theiruse with parts of the human body.From the perspective of MT, translating per-sonal pronouns from English to morphologicallyricher languages, such as French (Le Nagard andKoehn, 2010), German (Hardmeier and Federico,2010) and Czech (Guillou, 2012) has recentlyaroused higher interest. In these languages, oneusually has to ensure agreement in gender andnumber between the pronoun and its direct an-tecedent, which requires a coreference resolver tobe involved.In this work, we make use of the English-to-Czech translation implemented within the Tec-toMT system (Zˇabokrtsky´ et al., 2008). In contrastto the phrase-based approach (Koehn et al., 2003),TectoMT performs a tree-to-tree machine transla-tion. An input English sentence is first analyzedinto its deep-syntactic representation, which issubsequently transferred into Czech. The pipelineends with generating a surface form of the Czechtranslation from its deep representation.The deep syntactic representation of a sentencein TectoMT follows the Prague tectogrammaticstheory (Sgall, 1967; Sgall et al., 1986). It is a de-pendency tree whose nodes correspond to contentwords. Personal pronouns missing on the surfaceare reconstructed in special nodes. All nodes areassigned semantic roles and coreference relationsare annotated.Originally, translation of both it and reflexivepronouns was treated by rules in TectoMT. TheEnglish deep representation of it was translated asto and a simple heuristics determined if it is be-ing expressed on the surface. Similarly, reflexiveswere always translated as se. This paper evalu-ates the translation quality reached using special-ized classifiers for these pronouns. Unlike the re-lated work on pronouns in MT, we focus on im-proving the lexical choice, not tuning other com-ponents that affect generating a particular surfaceform (e.g. coreference resolution).2 Linguistic analysisWe started with an analysis of how the pronounsunder investigation are translated1 in two Czech-English parallel treebanks – Prague Czech-EnglishDependency Treebank 2.0 (Hajicˇ et al., 2011,PCEDT) and CzEng 1.0 (Bojar et al., 2012).1Note that besides the means mentioned below, there areother ways of translating these pronouns. However, in mostcases they can be replaced by one of the variants listed withno harm to the quality of the Czech output.1037English Czechemphasis on objectobject aﬀected by its own action reﬂexive pronoun sepronoun sámboth sám and seadjective samotnýcoreferential with VP or a segmentdemonstrative pronoun tocoreferential with NP personal pronounpleonastic nothing, structure changes[PersPron][To][Null][Samotny][Sam][Se][SamSe]ItReflexivesFigure 1: The mapping of the types of English it (top) andreflexive pronouns (bottom) to their Czech counterparts.2.1 Translating it from English to CzechIn English, three coarse-grained types of it are tra-ditionally distinguished: referential it pointing to anoun phrase in the preceding or following context,anaphoric it referring to a verbal phrase or a largerdiscourse segment, and non-referential pleonasticit, whose presence is imposed only by the syntac-tic rules of English.There are three prevailing ways of translat-ing it into Czech, also three different ways pre-vail. Personal pronouns or zero forms2, whosegender and number are determined by their an-tecedent, are the most frequent variant (referred toas the PersPron class in the following). Anotherway is using the Czech demonstrative pronoun to,which is a neuter singular form of the pronoun ten(To class). The third option results in fact no lex-ical counterpart in the Czech translation, the En-glish and Czech sentences thus having a differentsyntactic structure (Null class).The mapping between English and Czech typesis shown in Figure 1. The To class is particu-larly overloaded. Even if a given occurrence ofit corefers with a noun phrase, translating it to todoes not require identifying the antecedent sincethe gender and number of to are always fixed (seeExample 1).(1) Some investors say Friday’s sell-off was agood thing. “It was a healthy cleansing,”says Michael Holland.Neˇkterˇı´ investorˇi rˇı´kajı´, zˇe pa´tecˇnı´vy´prodej byla dobra´ veˇc. “Byla to zdrava´ocˇista,” rˇı´ka´ Michael Holland.2Czech is a pro-drop language.2.2 Translating reflexive pronouns fromEnglish to CzechAccording to the Longman Dictionary of Contem-porary English,3 reflexive pronouns are typicallyused in two scenarios: to show that the object isaffected by its own action and to emphasize thatthe utterance relates to one particular thing, per-son etc. (see Example 2).(2) The Gambia’s President himself partici-pated in the hunt last year.The most usual Czech counterparts of Englishreflexives comprise the Czech reflexive pronounse (Se class), the adjective samotny´ (Samotnyclass) and the pronoun sa´m (Sam class), all in var-ious morphological forms. Moreover, sa´m oftenappears with se to emphasize that the action af-fecting the object is performed by the object itself(SamSe class). Figure 1 illustrates the correspon-dence between English usages and Czech expres-sions.3 DataTo train and intrinsically evaluate TMs for it andEnglish reflexives, we have extracted data from theentire PCEDT and 11 sections of CzEng. Bothtreebanks follow the annotation style based on thePrague tectogrammatics theory (Sgall, 1967; Sgallet al., 1986). While PCEDT consists of 50,000sentence pairs annotated mostly manually, the an-notation of CzEng with 15 million parallel sen-tences is entirely automatic. Both treebanks havebeen provided with a fully automatic alignment ofCzech and English nodes (Marecˇek et al., 2008),which is, however, prone to errors for it and itsCzech counterparts. Since they are pronouns, theycan replace a wide range of content words andtheir meaning is inferred mainly from the context.The situation is better for verbs as their usual par-ents in dependency trees: since they carry meaningin a greater extent, their automatic alignment is ofa higher quality.We took advantage of this property and the goldannotation of semantic roles in PCEDT, obtainingCzech translations as the argument of the Czechverb aligned with the English parent verb that fillsthe same semantic role as the given it. Using thisapproach, we succeeded in reaching the Czechcounterpart in more than 60% of instances. Therest had to be done manually.3http://www.ldoceonline.com1038It Train Test Reflexives Train TestPCEDT sections 00–19 20–21 CzEng sections 00–09 98PersPron 576 322 Se 6,305 652To 231 138 Sam 2,271 205Null 133 83 SamSe 1,361 129Samotny 804 89Total 940 543 Total 10,741 1,075Table 1: Distribution of classes in the data sets.Czech counterparts of English reflexive pro-nouns have been collected directly from the align-ment in CzEng, ignoring the cases where thealigned Czech word does not fall in one of theclasses mentioned in Section 2.2.The overall statistics of the train and the test setare shown in Table 1. The disproportion of train-ing instances for it results from the manual anno-tation of classes, which could not be completelyfinished due to time reasons. In order to maintainthe overall distribution, we also had to limit thenumber of automatically annotated classes.Given the observation (see Section 2), we de-signed features to differentiate between the waysit and reflexives are translated.3.1 Features for itThe translation mapping in Figure 1 suggests thatidentifying the English type of it might be infor-mative. We thus constructed a binary coreference-related feature based on the output of the systemNADA (Bergsma and Yarowsky, 2011) giving anestimate of whether an instance of it is coreferen-tial.Some verbs are more likely to bind with itthat refers to a longer utterance. Such it is rel-atively consistently translated as a demonstrativeto. However, PCEDT is too small to be a suffi-cient sample from a distribution over lexical prop-erties. Hence, we took advantage of CzEng andcollected co-occurrence counts between a seman-tic role that the given it fills concatenated with alemma of its verbal parent and a Czech counter-part having the same semantic role (denoted ascsit). We filtered out all occurrences where csitwas neither a personal pronoun nor to. For bothpossible values of csit a feature is constructed bylooking up frequencies for a concrete occurrencein the co-occurrence counts collected on CzEngand quantized into 4-5 bins following the formula:bin(log(count(semrole : parent ∧ csit)count(semrole : parent)count(csit))).Linguistic analysis suggested including syntax-beitsubj obj adj:compl v:to+infv:that+ﬁnbeitsubja) b)Figure 2: Examples of syntactic features capturing typicalconstructions with a verb be.oriented patterns related to the verb to be such asthose shown in Figure 2. For instance, nominalpredicates4 tend to be translated as to even if itis coreferential. On the other hand, an adjectivalpredicate followed by a subordinating clause in-troduced by the English connectives to or that usu-ally indicates a pleonastic usage of it translated asa null subject.3.2 Features for reflexive pronounsHere we focused on distinguishing between thetwo most frequent meanings (see Section 2.2).Ideally, the POS tag of the parent would be asufficient feature because reflexives in the secondmeaning should depend on a noun. However, sincewe deal with automatically parsed trees we had tosupport the parent POS tag by the POS tag of theimmediately preceding word. Moreover, anotherfeature indicates if the preceding word is a nounand agrees with the pronoun in gender and num-ber.Furthermore, we observed that sa´m rarely ap-pears in other case than nominative. Although thisfeature exploits the target side, we can use it sincethe case of the governing Czech noun is alreadyknown at the point when reflexives are translated.Last but not least, the morpho-syntactic pattern(including a possible preposition) in which the re-flexive pronoun appears is a valuable feature.4 Experiments and EvaluationTo mitigate a possible error caused by a wrongclassifier choice, we built several models based onvarious Machine Learning classification methodsincluding Maximum Entropy inplemented in theAI::MaxEntropy Perl library,5 logistic regressionwith one-against-all strategy from Vowpal Wab-bit6 as well as decision trees, k-NN and SVM fromScikit-learn library (Pedregosa et al., 2011).4The verb to be has an object.5http://search.cpan.org/˜laye/AI-MaxEntropy-0.206http://hunch.net/˜vw1039It ReflexivesTrain Test Train TestBaseline 60.70 59.30 58.70 60.65AI::MaxEntropy 85.99 76.61 76.37 77.77VW (passes=20, l2=10e-5) 89.99 76.43 76.98 77.77sklearn:decision-trees 93.36 73.66 81.78 76.37sklearn:k-NN (k=10) 82.51 73.30 77.64 76.74sklearn:SVM (kernel=linear) 90.83 75.51 76.55 78.14Table 2: Accuracy of both translation models on the trainingand test data.We compare our results with a majority classbaseline (PersPron and Se classes) in Table 2.The results show a 17% gain when our approachis used.The specialized models have been integrated inthe TectoMT system and extrinsically evaluatedon the English-Czech test set for the WMT 2011Translation Task (Callison-Burch et al., 2011).7This data set contains 3,003 English sentenceswith one Czech reference translation, out of which430 contain at least one occurrence of it and 52contain a reflexive pronoun.The new approach was compared to the origi-nal TectoMT rule-based pronoun handling heuris-tics (see Section 1). The shift from the originalsettings to the new translation models results in166 changed sentences with it and 17 changed sen-tences with English reflexives. In terms of BLEUscore, we observe a marginal drop from 0.1404 to0.1403 using the new approach. However, BLEUmay be too coarse for this kind of experiment.In order to give a more realistic view, we car-ried out a manual evaluation. All 17 modifiedsentences for reflexives and 50 randomly sampledchanged sentences containing it were presented toone annotator who assessed which of the two sys-tems gave a better translation. Table 3 shows thatimproved sentences dominate in both cases. Over-all, the improved sentences account for around8.5% of all sentences with it and 23% sentencescontaining a reflexive pronoun.5 DiscussionLooking into the types of improvements and er-rors in the manually evaluated sentences, we havefound that the new model for it opted for a dif-ferent translation only in cases where the originalsystem decided to express to on the surface. In13 out of 24 improvements, the new model for itsucceeded in correctly resolving the Null class7http://www.statmt.org/wmt11/test.tgzIt Reflexivesnew better than old 24 12old better than new 13 0equal quality 13 5Table 3: The results of manual evaluation on sentences trans-lated by TectoMT in the original settings and using the newtranslation modelswhile in the remaining 11 cases, the correctedclass was PersPron. It took advantage mostlyof the syntax-based features in the former case andthe coreference-related feature in the latter.Regarding the reflexive pronouns the pronounwas used in its emphasizing meaning in all but twoaltered sentences. This accords with the designof features, which are mainly targeted at revealingthis usage of reflexives. Moreover, the feature in-dicating if a Czech noun is in nominative case hasproved to be particularly useful, correctly drivingthe lexical choice between sa´m and samotny´. Themajority of errors stem from incorrect activationof syntactic features due to parsing and POS tag-ging errors.6 ConclusionsIn this work, we presented specialized translationmodels for two types of English pronouns: it andreflexives. Integrating them into an English-Czechsyntax-based MT system TectoMT we succeededin improving the concerned sentences measuredby human evaluation.Generally, it is intractable to design a specificfeature set for every word. However, this workshows on two examples that the correct transla-tion of some words depends on many linguisticaspects, e.g. syntax and coreference and that isworth taking these aspects into account.AcknowledgmentsThis work has been supported by the GrantAgency of the Czech Republic (grantsP406/12/0658 and P406/2010/0875), the grantGAUK 4226/2011 and EU FP7 project Khresmoi(contract no. 257528). This work has been usinglanguage resources developed and/or stored and/ordistributed by the LINDAT-Clarin project of theMinistry of Education of the Czech Republic(project LM2010013).1040ReferencesShane Bergsma and David Yarowsky. 2011. NADA:A Robust System for Non-Referential Pronoun De-tection. In DAARC, pages 12–23, Faro, Portugal,October.Ondrˇej Bojar, Zdeneˇk Zˇabokrtsky´, Ondrˇej Dusˇek, Pe-tra Galusˇcˇa´kova´, Martin Majlisˇ, David Marecˇek, Jirˇı´Marsˇı´k, Michal Nova´k, Martin Popel, and Alesˇ Tam-chyna. 2012. The Joy of Parallelism with CzEng1.0. In Proceedings of LREC 2012, Istanbul, Turkey,May. ELRA, European Language Resources Associ-ation.Chris Callison-Burch, Philipp Koehn, Christof Monz,and Omar Zaidan. 2011. Findings of the 2011Workshop on Statistical Machine Translation. InProceedings of the Sixth Workshop on Statisti-cal Machine Translation, pages 22–64, Edinburgh,Scotland, July. Association for Computational Lin-guistics.Liane Guillou. 2012. Improving Pronoun Translationfor Statistical Machine Translation. In Proceedingsof the Student Research Workshop at the 13th Con-ference of the EACL, pages 1–10, Stroudsburg, PA,USA. Association for Computational Linguistics.Jan Hajicˇ, Eva Hajicˇova´, Jarmila Panevova´, Petr Sgall,Silvie Cinkova´, Eva Fucˇı´kova´, Marie Mikulova´, PetrPajas, Jan Popelka, Jirˇı´ Semecky´, Jana Sˇindlerova´,Jan Sˇteˇpa´nek, Josef Toman, Zdenˇka Uresˇova´, andZdeneˇk Zˇabokrtsky´. 2011. Prague Czech-EnglishDependency Treebank 2.0.Christian Hardmeier and Marcello Federico. 2010.Modelling Pronominal Anaphora in Statistical Ma-chine Translation. In Marcello Federico, Ian Lane,Michael Paul, and Franc¸ois Yvon, editors, Proceed-ings of the seventh International Workshop on Spo-ken Language Translation (IWSLT), pages 283–289.Philipp Koehn, Franz Josef Och, and Daniel Marcu.2003. Statistical Phrase-based Translation. In Pro-ceedings of the 2003 Conference of the NAACL HLT– Volume 1, pages 48–54, Stroudsburg, PA, USA.Association for Computational Linguistics.Ronan Le Nagard and Philipp Koehn. 2010. Aid-ing Pronoun Translation with Co-Reference Resolu-tion. In Proceedings of the Joint Fifth Workshop onStatistical Machine Translation and MetricsMATR,pages 252–261, Uppsala, Sweden, July. Associationfor Computational Linguistics.David Marecˇek, Zdeneˇk Zˇabokrtsky´, and Va´clavNova´k. 2008. Automatic Alignment of Czech andEnglish Deep Syntactic Dependency Trees. In Pro-ceedings of the Twelfth EAMT Conference, pages102–111.Izak Morin. 2009. Translating Pronouns, ProperNames and Kinship Terms from Indonesian into En-glish and vice versa. TEFLIN Journal: A publica-tion on the teaching and learning of English, 16(2).Kristy´na Onderkova´. 2010. Possessive Pronouns inEnglish and Czech Works of Fiction, Their Use withParts of Human Body and Translation. Master’s the-sis, Masaryk University, Faculty of Arts.Fabian Pedregosa, Gae¨l Varoquaux, Alexandre Gram-fort, Vincent Michel, Bertrand Thirion, OlivierGrisel, Mathieu Blondel, Peter Prettenhofer, RonWeiss, Vincent Dubourg, Jake Vanderplas, Alexan-dre Passos, David Cournapeau, Matthieu Brucher,Matthieu Perrot, and E´douard Duchesnay. 2011.Scikit-learn: Machine Learning in Python. Jour-nal of Machine Learning Research, 12:2825–2830,November.Petr Sgall, Eva Hajicˇova´, and Jarmila Panevova´. 1986.The Meaning of the Sentence in Its Semantic andPragmatic Aspects. D. Reidel Publishing Company,Dordrecht.Petr Sgall. 1967. Generativnı´ popis jazyka a cˇeska´deklinace. Academia, Prague, Czech Republic.Zdeneˇk Zˇabokrtsky´, Jan Pta´cˇek, and Petr Pajas. 2008.TectoMT: Highly Modular MT System with Tec-togrammatics Used as Transfer Layer. In Proceed-ings of the Third Workshop on Statistical MachineTranslation, pages 167–170, Stroudsburg, PA, USA.Association for Computational Linguistics.1041

Two Case Studies on Translating Pronouns in a Deep Syntax Framework

http://aclweb.org/anthology//I/I13/I13-1142.pdf

Two Case Studies on Translating Pronouns in a Deep Syntax Framework

Abstract

Similar works

Full text

Available Versions

Biblio at Institute of Formal and Applied Linguistics