Chinese and Spanish are the most spoken languages in the world. However, there is not much research done in machine translation for this language pair. We experiment with the parallel Chinese-Spanish corpus (United Nations) to explore alternatives of SMT strategies which consist on using a pivot language. Particularly, two well-known alternatives are shown for pivoting: the cascade system and the pseudo-corpus. As Pivot language we use English, Arabic and French. Results show that English is the best pivot language between Chinese and Spanish. As a new strategy, we propose to perform a combination of the pivot strategies which is capable to highly outperform the direct translation strategy.Postprint (published version

Banchs, Rafael E.

Henríquez, Carlos

Ruiz Costa-Jussà, Marta

English

Chinese and Spanish are the most spoken languages in the world. However, there is not much research done in machine translation for this language pair. We experiment with the parallel Chinese-Spanish corpus (United Nations) to explore alternatives of SMT strategies which consist on using a pivot language. Particularly, two well-known alternatives are shown for pivoting: the cascade system and the pseudo-corpus. As Pivot language we use English, Arabic and French. Results show that English is the best pivot language between Chinese and Spanish. As a new strategy, we propose to perform a combination of the pivot strategies which is capable to highly outperform the direct translation strategy

UPCommons

Enhancing scarce-resource language translation through pivot combinations

UPCommons. Portal del coneixement obert de la UPC

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1361–1365,Chiang Mai, Thailand, November 8 – 13, 2011. c 2011 AFNLPEnhancing scarce-resource language translation through pivot combinationsMarta R. Costa-jussa`Barcelona Media Innovation CenterAv. Diagonal, 17708018 Barcelonamarta.ruiz@barcelonamedia.orgCarlos Henrı´quezTALP-UPCJordi Girona, s/n08034 Barcelonacarlos.henriquez@upc.eduRafael E. BanchsInstitute for Infocomm Research1 Fusionopolis Way 21-01Singapore 138632rembanchs@i2r.a-star.sg.eduAbstractChinese and Spanish are the most spoken lan-guages in the world. However, there is notmuch research done in machine translation forthis language pair. We experiment with theparallel Chinese-Spanish corpus (United Na-tions) to explore alternatives of SMT strate-gies which consist on using a pivot language.Particularly, two well-known alternatives areshown for pivoting: the cascade system andthe pseudo-corpus. As Pivot language we useEnglish, Arabic and French. Results showthat English is the best pivot language betweenChinese and Spanish. As a new strategy, wepropose to perform a combination of the pivotstrategies which is capable to highly outper-form the direct translation strategy.1 IntroductionAlthough they are very distant languages, Chineseand Spanish are very close to each other in the rank-ing of the most spoken languages in the world1.Nevertheless, when interested in bilingual resourcesbetween these two languages they become far apartagain. Similarly, the related amount of work wehave found within the computational linguistic com-munity, can be reduced to a very small set of refer-ences.The most popular research event recently per-formed was the 2008 IWSLT evaluation campaign 2.This evaluation organized two Chinese-to-Spanishtracks. One of them was focused on direct trans-lation and the other one on pivot translation through1www.ethnologue.org/ethno docs/distribution.asp?by=size2http://mastarpj.nict.go.jp/IWSLT2008/English. Best translation results were obtained byfar in the pivot task. The best system in the pivottask (Wang et al., 2008) compared two differentapproaches: The first one, training two translationmodels on the Chinese-English corpus and English-Spanish corpus, and then building a pivot translationmodel for Chinese-Spanish translation using Englishas a pivot language as proposed in (Wu and Wang,2007); the second one obtained better results and itwas based on a cascade approach. The idea here isto translate from Chinese into English and then fromEnglish to Spanish, which means performing twotranslations. Besides the research mentioned above,which directly addressed the Chinese-Spanish lan-guage pair, we may also find in the literature anotherapproach similar to Wu’s (2007) authored by Cohnand Lapata (2007). Basically, they also used sev-eral intermediate pivot language to create source-to-target phrases that are lately interpolate with a directsystem build with a source-to-target parallel corpus.Apart from the BTEC3 corpus available throughthe IWSLT4 competition andHoly Bible datasets de-scribed in (Paul, 2008) and (Banchs and Li, 2008),respectively, there is a recent release of a six lan-guage parallel corpus (including both Chinese andSpanish) from United Nations (UN) for researchpurposes (Rafalovith and Dale, 2009). Using therecently released UN parallel corpus as a startingpoint, this work focuses on the problem of develop-ing Chinese-Spanish phrase-based SMT technolo-gies with a limited set of bilingual resources. Weexplore and evaluate different alternatives for the3Basic Traveller Expressions Corpus4International Workshop on Spoken Language Translation1361problem in hand by means of pivot-language strate-gies through the other languages available in the UNparallel corpus, such as Arabic, English and French.More specifically, strategies such as system cascad-ing and pseudo-corpus generation are implementedand compared against a baseline system implement-ing a direct translation approach. We propose a sys-tem combination different from previous ones (Wuand Wang, 2009) and based on the Minimum BayesRisk (MBR) (Kumar and Byrne, 2004) technique us-ing both pivot strategies which is capable to highlyoutperform the direct system. To the best of ourknowledge, this idea was not explored before andit is a way of increasing the quality of translationbetween languages with scarce bilingual resources.In addition, we are performing a combination ofthe same system but introducing new informationthrough the pivot language.The paper is structured as follows. Section 2 de-scribes the main strategies for performing Chinese-to-Spanish translation which are tested in this work.Section 3 presents the evaluation framework. Then,section 4 reports the experiments (including the sys-tem combination) and the results. Finally, section 5concludes and proposes new research directions.2 Direct and pivot statistical machinetranslation approachesThere are several strategies that we can follow whentranslating a pair of languages in Statistical MachineTranslation. The next three sub-sections present thedetails of the ones we are using in this work.2.1 Direct systemOur direct system uses the phrase-based translationsystem (Koehn et al., 2003). This popular systemimplements a log-linear model in which a sourcelanguage sentence fJ = f1, f2, . . . , fJ is trans-lated into another language (target) sentence eI =e1, e2, . . . , eI by searching for the translation hy-pothesis eˆI maximizing a log-linear combination ofseveral feature models (Och, 2003).The main systemmodels are the translation modeland the language model. The first one deals with theissue of which target language phrase fj translatesa source language phrase ei and the latter modelestimates the probability of translation hypothesis.Apart from these two models, there are other stan-dard models such as the lexical models, the wordbonus, and the reordering model.For decoding, we used the MOSES toolkit (Koehnet al., 2007) with the option of Minimum Bayes Risk(MBR) (Kumar and Byrne, 2004) decoding. There-fore the 1best translation obtained is not the one withhighest priority but the one that is most similar tothe most likely translation. The option was activatedwith its default parameters so it considered the top200 distinct hypothesis to compute the 1best.2.2 Cascade SystemThis approach handles the source-pivot and thepivot-target system independently. They are bothbuilt and tuned to improve their local translationquality and then joined to translate from the sourcelanguage to the target language in two steps: first,the 1best translation output from source to pivot iscomputed and, second, it is used to obtain the 1besttarget translation output as the final translation.There is an alternative approach that considers thenbest list in each step instead of the 1best. For in-stance, it was used in (Khalilov et al., 2008) withtheir cascade approach in order to obtain the bestChinese-Spanish translation. We also implementedit but the results were similar that those using MBRdecoding in each system and keeping the 1best trans-lation. Therefore we maintained MBR decoding forthe rest of the experiments, which is also easier towork with.2.3 Pseudo-Corpus SystemThis approach translates the pivot section of thesource-pivot parallel corpus to the target languageusing a pivot-target system built previously. Then, asource-target SMT system is built using the sourceside and the translated pivot side of the source-pivotcorpus. The pseudo-corpus system is tuned using adirect source-target development corpus.2.4 Pivot combinationUsing the 1-best translation output from the differ-ent pivot strategies, we built an N-best list and com-puted the final translation using MBR. MBR hasbeen used both during decoding (Kumar and Byrne,2004; Ehling et al., 2007) and as a postprocess overan N-best list. The current version of the MOSES1362toolkit includes both MBR implementations. For thesystem combinations we used the second one.The MBR algorithm implemented in MOSES uses(1 − BLEU)β as the Loss Function. The value βweights the hypothesis proportionally to its transla-tion score, but we considered all our hypothesis asequal so β was a constant and therefore could bediscarded. At the end, MBR choose the hypothesesE′that fulfills:E′= argminEˆ′   XE6=Eˆ′1 − BLEU(E, Eˆ′)   (1)It is important to mention that all N-best list musthave at least 3 hypothesis per sentence. Having onlytwo hypothesis would not work as expected becausethe Loss Function would always choose the longestone, which can be explained by the definition ofBLEU:BLEU(E,E′) =exp NXn=1logpn(E,E′)N!∗  (E,E′) (2)where pn(E,E′) is the precision of n-grams in thehypothesis E′with reference E; and  (E,E′) is abrevity penalty if the hypothesis E′ is shorter thanthe reference E. Then pn(E,E′) = pn(E′, E) and?E,E′ : length(E) > length(E′) :1 − BLEU(E,E′) ≥ 1 − BLEU(E′, E) (3)3 Evaluation FrameworkThis section introduces the details of the evaluationframework used. We report the UN corpus statistics,a description of how we built the systems and theevaluation details.3.1 Corpus statisticsIn this study we use the UN corpus taking advantageof the fact that (as far as we are concerned) it is thebiggest parallel corpus freely-available in Chinese-Spanish and it contains the same sentences in sixother languages, therefore we can experiment withdifferent pivot languages.When experimenting with different pivot lan-guages, in order to make the systems as compara-ble as possible, we first did a sentence selection overthe corpus so all systems were built exactly withthe same training, tuning and testing sets. All cor-pora were tokenized, using the standard MOSES to-kenizer for Spanish, English and French; ictclass(Zhang et al., 2003) for Chinese; and MADA+TO-KAN (Habash and Rambow, 2005) for Arabic. TheSpanish, English and French corpora were lower-cased. If a sentence had more than 100 words inany language, it was deleted from all corpora. Ifa sentence pair had a word ratio bigger than threefor any Chinese-Pivot or Pivot-Spanish parallel cor-pora, it was deleted from all corpora. For all lan-guages, we identify all sentences that occur onlyonce in the corpora. The tuning and testing setswhere drawn from the available multilingual cor-pus by using a maximum perplexity and lowest out-of-vocabulary word criterion over the English partof the dataset. In order to do this, perplexity wascomputed on a sentence-by-sentence basis by usinga leave-one-out strategy; then, we selected the twothousand sentences which had the highest perplex-ity and the lowest out-of-vocabulary words for con-structing the tuning and testing sets. Table 1 showsthe main statistics for all corpora.training development tests w s w s wZh 58.6k 1.6M 1k 30.9k 1k 32.6kEs 58.6k 2.3M 1k 42.2k 1k 44.0kEn 58.6k 2.0M 1k 36.7k 1k 38.3kAr 58.6k 2.6M 1k 47.9k 1k 49.9kFr 58.6k 2.3M 1k 42.1k 1k 43.9kTable 1: UN Corpus Statistics (s stands for number ofsentences and w for number of words)3.2 System detailsOur systems were build using MOSES. For allsystems, we used the default MOSES parameters,which includes the grow-final-diagonal alignmentsymmetrization, the lexicalized reordering, a 5-gramlanguage model using Kneser-Ney smoothing andphrases up to length 10. The optimization was doneusing MERT (Och, 2003). The decoding was doneusing MBR.13634 Chinese-to-Spanish MT strategiesGiven the different languages available in the UNcorpora, we tested three different pivot languages.Additionally, we compared the cascade and thepseudo-corpus pivot strategies. Finally, we com-bined the system outputs.4.1 Experimenting with different pivotlanguagesUsing most of the languages available in the UNparallel corpora (English, Spanish, Chinese, Arabicand French) we built and compare several translationsystems in order to study the impact of the differ-ent pivot languages when translating from Chineseto Spanish.Specifically, we built seven Chinese-Spanish sys-tems: the direct Chinese-Spanish system as a qual-ity upper bound; three cascade approach and threepseudo-corpus, using English, Arabic and French aspivots.Therefore, the first step was to build the differentChinese-Pivot and Pivot-Spanish systems. Table 2shows the BLEU achieved with the intermediate sys-tems trained with the UN Corpus. These systems areused in the next section when experimenting withdifferent pivot languages.BLEUChinese-English 35.67Chinese-Arabic 46.11Chinese-French 28.31English-Spanish 51.64Arabic-Spanish 41.79French-Spanish 46.42Table 2: UN Pivot SystemsAs we can see in Table 2 the best Chinese-Pivotsystem is the Chinese-Arabic system. As for thePivot-Spanish system, the one that achieved the bestBLEU score was the English-Spanish system.Tables 3 shows the results for our Chinese-Spanish configurations with the UN corpus. We cansee there that the best pivot system used the cascadeapproach with English as the pivot language.The fact that the pseudo-corpus through En-glish outperforms cascade through English is notstatistically significant, with a 95% confidenceLanguages System BLEUChinese-Spanish direct 33.06Chinese-English-Spanish cascade 32.90Chinese-French-Spanish cascade 30.37Chinese-Arabic-Spanish cascade 28.88Chinese-English-Spanish pseudo 32.97Chinese-French-Spanish pseudo 32.61Chinese-Arabic-Spanish pseudo 32.23Table 3: UN pivot languages. Best results in bold.(Koehn, 2004). These results, however, are coherentwith previous works using the same language pair(Bertoldi et al., 2008; Henrı´quez Q. et al., 2010) thatalso reported the pseudo-corpus strategy was betterthan the cascade strategy.In all cases English is statistically significant thebest pivot language, with a 99% confidence, whichis coherent with the Pivot-Spanish results in table 2.Further analysis is required in order to understandwhy the cascade through English is able to help somuch in the Chinese-to-Spanish translation.4.2 Pivot combinationTable 4 shows the results of the different output sys-tems combined (from table 3) with the MBR tech-nique. En + Ar + Fr Cascade + Pseudo (whichcombines all system outputs from table 3 exceptthe direct approach) is better than the Chinese-to-Spanish direct system and it is significant with a99% of confidence. When adding the direct ap-proach (dir) it increases the translation performanceslightly and we obtain the best Chinese-to-Spanishtranslation.casc pseudo casc+pseudoEn+Ar+Fr 32.66 33.30* 33.97*dir+En+Ar+Fr 33.60* 33.77* 34.09*Table 4: Output system combination usingMBR. * showsstatistically significantly better results than the direct sys-tem (with a 99% of confidence). Best results in bold.Casc stands for cascade.5 ConclusionsThis work has presented experimental research forthe Chinese-Spanish translation pair. The main con-1364clusions derived from our study are:• English is the best Pivot language for Chinese-to-Spanish compared to languages such asFrench or Arabic. The system built using En-glish as Pivot was significantly better than theones built with either French or Arabic, with a99% confidence in both cases.• There is not a significant difference amongthe best cascade and pseudo-corpus pivot ap-proaches.• The output combination using MBR is able toimprove the direct system in 1 BLEU point inthe best case. This improvement is significantlybetter with a 99% confidence.6 AcknowledgmentThe authors would like to thank Barcelona MediaInnovation Center and Institute for Infocomm Re-search for its support and permission to publish thisresearch. This work has been partially funded bythe Spanish Department of Science and Innovationthrough the Juan de la Cierva fellowship program.ReferencesR. Banchs and H. Li. 2008. Exploring spanish morphol-ogy effects on chinese-spanish smt. In MATMT 2008:Mixing Approaches to Machine Translation, pages 49–53, Donostia-San Sebastian, Spain, February.N. Bertoldi, R. Cattoni, M. Federico, and M. Barbaiani.2008. FBK @ IWSLT-2008. In Proc. of the Inter-national Workshop on Spoken Language Translation,pages 34–38, Hawaii, USA.T. Cohn and M. Lapata. 2007. Machine Translation byTriangulation: Making Effective Use of Multi-ParallelCorpora. In Proc. of the ACL.N. Ehling, R. Zens, and H. Ney. 2007. Minimum bayesrisk decoding for bleu. In Proc. of the ACL, pages101–104, Prague, Czech Republic, June. Associationfor Computational Linguistics.N. Habash and O. Rambow. 2005. Arabic tokeniza-tion, part-of-speech tagging and morphological dis-ambiguation in one fell swoop. In Proc. of the 43rdAnnual Meeting of the Association for ComputationalLinguistics, pages 573–580, Ann Arbor, MI, June.C. A. Henrı´quez Q., R. E. Banchs, and J. B. Marin˜o.2010. Learning reordering models for statistical ma-chine translation with a pivot language. Internal Re-port TALP-UPC.M. Khalilov, M. R. Costa-Jussa`, C. A. Henrı´quez, J. A. R.Fonollosa, A. Herna´ndez, J. B. Marin˜o, R. E. Banchs,B. Chen, M. Zhang, A. Aw, and H. Li. 2008. TheTALP & I2R SMT Systems for IWSLT 2008. In Proc.of the International Workshop on Spoken LanguageTranslation, pages 116–123, Hawaii, USA.P. Koehn, F. J. Och, and D. Marcu. 2003. Statisticalphrase-based translation. In HLT-NAACL, pages 48–54.P. Koehn, H. Hoang, A. Birch, C. Callison-Burch,M. Federico, N. Bertoldi, B. Cowan, W. Shen,C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin,and E. Herbst. 2007. Moses: Open source toolkit forstatistical machine translation. In Proc. of the ACL,pages 177–180, Prague, Czech Republic.P. Koehn. 2004. Statistical significance tests for machinetranslation evaluation. In Proceedings of EMNLP, vol-ume 4, pages 388–395.S. Kumar and W. Byrne. 2004. Minimum bayes-risk decoding for statistical machine translation. InProceedings of the Human Language Technology andNorth American Association for Computational Lin-guistics Conference (HLT/NAACL’04), pages 169–176, Boston, USA, May.F.J. Och. 2003. Minimum error rate training in statisticalmachine translation. In Proc. of the 41th Annual Meet-ing of the Association for Computational Linguistics,pages 160–167.M. Paul. 2008. Overview of the IWSLT 2008 Evalua-tion Campaign. In Proc. of the International Workshopon Spoken Language Translation, pages 1–17, Hawaii,USA.A. Rafalovith and R. Dale. 2009. United nations generalassembly resolutions: A six-language parallel corpus.In Proc. of the MT Summit XII, pages 292–299, Ot-tawa.H. Wang, H. Wu, X. Hu, Z. Liu, J. Li, D. Ren, andZhengyuNiu. 2008. The TCH Machine Transla-tion System for IWSLT 2008. In Proc. of the Inter-national Workshop on Spoken Language Translation,pages 124–131, Hawaii, USA.H. Wu and H. Wang. 2007. Pivot Language Approachfor Phrase-Based Statistical Machine Translation. InProc. of the ACL, pages 856–863, Prague.H. Wu and H. Wang. 2009. Revisiting Pivot LanguageApproach for Machine Translation. . In Proc. of theACL-IJCNLP, pages 154–162, Singapore.H. Zhang, H. Yu, D. Xiong, and Q. Liu. 2003. HHMM-based chinese lexical analyzer ICTCLAS. In Proc. ofthe 2nd SIGHAN Workshop on Chinese language pro-cessing, pages 184–187, Sapporo, Japan, July.1365

https://upcommons.upc.edu/bitstream/2117/102471/1/Enhancing%20scarce-resource%20language%20translation%20through%20pivot%20combinations%20-%20I11-1154.pdf

Enhancing scarce-resource language translation through pivot combinations

Abstract

Similar works

Full text

Available Versions

UPCommons

UPCommons. Portal del coneixement obert de la UPC