2005-2006 > Academic research: refereed > Publication in refereed journalVersion of RecordPublishe

Chi, Z

Wang, R

2005-2006 > Academic research: refereed > Publication in refereed journalVersion of RecordPublishedVoR allowe

PolyU Institutional Repository

?　?　?　?　?　??19? ? 1?　 JOURNALOF CHINESE INFORMATION PROCESSING Vol.19 No.1????:1003-0077(2005)01-0021-09???????????????????①??? ,???(?????????????? ????????? ,??)??:句子相似度的衡量是基于实例机器翻译研究中最重要的一个内容。对于基于实例的汉英机器翻译研究 ,汉语句子相似度衡量的准确性 , 直接影响到最后翻译结果的输出。本文提出了一种汉语句子结构相似性的计算方法。该方法比较两个句子的词类信息串 , 进行最优匹配 ,得到一个结构相似性的值。在小句子集上的初步实验结果表明 ,该方法可行 , 有效 ,符合人的直观判断。???:人工智能;机器翻译;基于实例机器翻译;汉英机器翻译;句子相似度衡量;自然语言处理?????:TP391　　　?????:AA Similarity Measure Method of Chinese Sentence StructuresWANG Rong-bo , CHI Zhe-ru(Center for Multimedia Signal Processing , Department of Electronic and InformationEngineering ,The Hong Kong Polytechnic University ,Hong Kong , China)Abstract:Example-based machine translation(EBMT)is an important branch of machine translation that has been studied ex-tensively for about twenty years.So far , some progresses have been gained because of researchershard work.Sentence simi-larity measure certainly is one of the most important problems addressed in EBMT.For EBMT from Chinese to English , theperformance of similarity measure of Chinese sentences affects directly final translation result of an input sentence.In this pa-per , we proposed a similarity measure method of Chinese sentence structures for example-based Chinese to English machinetranslation.In this method , the algorithm performs the optimal matching between the word type sequences of two compared sen-tences.The preliminary experimental results show that the measure method works well when it is tested on a small dataset.Key words:artificial intelligence;machine translation;example-based machine translation;Chinese-English machinetranslation;sentence similarity measure;natural language processing1　?????????? ,????????????? ,?? ,??????????? 。????? ,???????????????? ,?????? ,???????。????????(EBMT)??? ,?????????????????? ,?? ,??MT ????????[ 1～ 4]???????????? EBMT ????? 。??????????????:???????????? ,??????????????(??)?????????????????? ,??????????? ,????????????????? ,?????????? ,??????? ,????????21① ????:2004-05-26????:???(1978—),? , ??? ,???????????? , ???? , ????.?。? EBMT ? ,?????????????????? ,???????????? ,?????????? 。??????????? ,????????????????????????????。????? ,???????????? 。????????????????????????[ 5] 。???????????:???????????????/???????。??????? ,????????? ,?????????????。?? ,?????????? ,??????????????????。????/?????????????????????? ,?????????????????? ,?????????。???[ 7]? ,?????????????????? 。????? ,??????????? ,??????????????????????????。???????? ,?? ,???????? 。???? ,??????????????????????????????? ,???????????????。???????????????????????? ,??????????? ,???????????????????? ,???????????????????????????? 。??????????? ,????? ,???????????。?????????????????????????????[ 6] 。?????????? ,???[ 8]? ,??????????????????????????? ,??????????????? 。???????????????????????????[ 9] 。?? ,?????????????????????[ 10] 。????? ,?????????????????。?????????????? ,?????????????????????? 。??? ,???????????????????????????????? ,??????????????。?? ,??????????????? ,???? ,???????? ,???????? 。?????????????????????????????????? 。????????: 2cm+n (1)?? m , n ???????????。? c???????????? 。????????????? 。??????????????????。????????????????? ,????????????????? ,???????????????? 。???? ,??????????????? ,?????????????? ,????????? 。?????????????????。??? ,?????????????????? ,????????????? ,???????????????。????????????????????。????????????????? ,????????????? ,????????????????????。?????? ,??????? ,????? 。2　?????2.1　???????????????? ,????????? 。22(1)?????????????? ,???????????? 。?? ,?????????????? ,????????????。(2)???????????? ,???????????? ,???????????? 。?? ,?????????????? ,????????????。???? ,??????????????。2.2　?????????????????:???????????? ,????????? ,???? ,????????? ,?????????????????????????。2.3　???? 1:????(Joint Condition):???????? ,????????? 。?? 2:????(JointNode):???????? ,???????????????? ,????????? 。?? 3:???? 1(CurrentNode1):????????? ,?????????? 。???? 2(CurrentNode2):????????? ,?????????? 。?? 4:??????(CurrentJointNode):?????????? ,???(????)??????????????。?? 5:?????(Direct Graph Expression):??????????? ,?????????????????????????????????。????????????????? 。?????“ ＊”?? ,???“ #”?? ,????????0? w0 。??????????:? 1　??????? ,?????????????(n≥m),?????? S1 ? S 2。S1:s1 s2 s3…si …sn-2 sn-1 sn.23S2:t 1 t 2 t 3…tj …tm-2 tm-1 tm .?? , si(1≤i≤n)???? S 1??? i ????? , tj(1≤j≤m)???? S 2 ??? j?????。???????? ,????(S1 , S2)????????????。? 2　???? S1 ? S2 ?????(????????)??? 2 ,???????????????? 。?????????????????? 。???????????? ,???????????????。???????:(1)???????。?? ,????????????。???? ,?????????? ,????????。?? ,???????? ,??????????? ,???????? 。???????? ,????????????? ,????? 。? 3　????????? 4　????????(2)?????????? 。??????? ,?????????? ,??????????。a.????????(S2)???????????(??“ #”??),?????? 1 ,???????(S1)?????(??“ #”??)?????? 2 ,?? S1 ???“ #”???????? 。?????? 1?????? ,???????? 。?? ,????????? 。b.??????1???“ ＊” ,???。?????? 1????? ,???????????????????? 2 ,????? e 。c.?????? 2???“ ＊” ,????? e 。?????? 2????? ,???????? ,????????????????2 ,????? e。??????2??????? ,??????1??????2 ,????????? ,?????1????? 2?? ,???????? 2???????? ,???? e 。?????? 2??????? ,??????1??????? 2 ,?????? d。24??????2?????? ,??????1??????2 ,???????? ,?????? 2??????? ,???? e。?????? 2?????? ,?????? 1??????? 2 ,???? d。d.??????2???????? ,??????? 2 ,???? c。e.???????? 1???????? ,??????? 1 ,???? b。???? 4?? ,?????? 。?4????????????????????。????? ,??????????? sn-3???????? tm-3?? , sn-1? tm-1??。????????????????? ,?????????????? ,??????????????????? 。?????????????? ,?????????(S 2)???????(tm),?????? 1 。????????? ,?????????? 。???????(S 1)?????(sn),??????2 ,?? S1???“ #”???????? 。??? tm? ,?????? 2??? s2 ,?????????? ,?????。????tm?????? tm-1 。????? tm-1?????(sn-1),???????????? ,?? sn-1???????? 。??? ,?? tm-2???? S 1??????????? ,? tm-3???? sn-3?? ,????。?????????????。? 5　???????????(???? 4)??????:k=m;1=n;CurrentNodel=tk;CurrentNode2=s1CurrentJointNode=“ #”IF(CurrentNodel==jointNode)THEN STOP;ELSE{WHILE(CurrentNode1 ! =“ ＊”){　IF(CurrentNodel==JointNode){CurrentNodel=CurrentNodel;CurrentNode2=CurrentNodel;}ELSE{TAG=1;WHILE(CurrentNode2! =` ＊' &&TAG==1){　IF(CurrentNode2==JointNode &&　CurrentNode2! =CurrentJoint Node){CurrentNode2=CurrentJoinNode;TAG=0;}ELSE IF(CurrentNode2==CurrentJointNode&&CurrentNode1==CurentNode2){　Cance1Joint(CurrentNode2);Joint(CurrentNodel , CurrentNode2);25CurrentJointNode=CurrentNode2;TAG=0;}　ELSE IF(CurrentNode2! =JointNode&&CurrentNode1==CurrentNode2){Joint(CurrentNode1 , CurrentNode2);CurrentJointNode=CurrentNode2;TAG=0;}ELSE IF((CurrentNode2==CurrentJointNode‖CurrentNode2! =JointNode)&&CurrentNode1! =CurrentNode2){1=1-1;CurrentNode2=s1;}　　　　　　　　}　　　　　}　　　　k=k-1;CurrentNode1=tk;}　　　}　??:?? Cance1Joint(), Joint()????????????????????????????? ,?????????。???????? ,??????????????:StruSim(s1 , s2)=2∑Ci=111 +∑Dij=1WjWi∑Ek=1Wk(2)　　???(2)? , C????????????(????)???。? Di????? ,????????。?????? ,???????? ,????? ,??????????????? ,?????????????????? 。???????? ,???????????? ,?????????“ ＊” 。?????????????? ,???????????。E ?????????????(??? ,????)。Wi ?? i ?????(??)???。Wj ???? j??????。Wk?????(??)?? k ?????? 。????????? 。(1)?????????? 。???????????????? ,? m=n , Di =0 , C=m , E =2m 。????(2)??1 ,???????????? 。(2)?????????? 。????????????????? ,?? ,??????? ,??????? 0 ,???(2)?? 0 ,???????????? 。2.4　?????????? ,???? 。26?1　?? 1:???????。　???:??/n??/n??/n ?/n 。/w?? 2:?????。 ???:??/n??/n?/n 。/w????????? 。S1:n n n n w.S2:n n n w.??????????????????? 。? 6　? 1 ????????????????????????? 7?? 。????????????????? 。　　?? 6?? , C=4 , D1=D2=D 3=0 ,D4 =1 , E=9.???? ,?????? 3????? ,???? 5????? ,??????? ,??????? 1。????????????? ,?????????????? 0.804。3　??????????? ,??????????????????????。?????? 3548????? ,????????????? 。?????????? 。1025　[ zj [ dj?/ r　[ dj ??/ s　[ ap ?/d　??/a　] ] ] 。/w　]1036　[ zj [ dj ?/ r　[ vp ?/v 　[ np ?/q　??/n　] [ vp [ pp ?/p　??/n ] ??/v　] ] ] 。/w　]1047　[ zj [ dj?/ r　[ vp?/d　[ vp ??/v　?/ r　[ np [mp ?/m　?/m　?/q　] ??/n　] ] ] ] 。/w　]1058　[ dj?/ r　[ vp??/ t　[ vp [ vp ?/v　?/ r　] [ vp [ vbar ?/v　?/u　] [ np [mp?/m　?/q　] ?/n　] ] ] ] ]1069　[ dj [ tp ??/v　?/f　] [ dj [ np ?/ r　?/c　??/n　] [ vp?/v　??/n　] ] ]??????? ,????????????? ,????? ,? vp , dj , v ??????????????? 。?? [` ' ? ]` ' ????? 。????? ,?? 31??? 。??????? ,?????????????? ,????3????? ,???? 5????? ,???? 1????????。??????? ,????????????????????(???)。?????????????????? ,?????????????。?????? ,? 1?????????。27? 1　????????????????????? ?????? ??????????? 1 ???????。????????????。 0.960 0.888?????。 0.818 0.675????。 0.600 0.000??????????。 0.375 0.041???????????????。 0.342 0.000??? 2 ????????。?????????????。 1.000 1.000??????????。 0.777 0.909???????????。 0.697 0.012????。 0.613 0.018????。 0.613 0.018??? 3 ??????。????????????。 1.000 0.247??????。 1.000 1.000??????????。 0.864 0.000?????。 0.854 0.015???????。 0.838 0.000??? 4 ????????。?????????????。 1.000 0.492????????。 1.000 1.000?????????。 0.869 0.006???????。 0.829 0.010??????????。 0.782 0.006　　?? 1? ,??????? 4?????????? 。???????? ,?????????? 5??????? 。?????????????????????????? 。??????????????????。??????? ,??????? ,??????????????????? 。?? ,?????? 3(?????? 。)? ,???????????????????????(??????? 。)???? ,???????? 1 ,????? ,??????????????(0.247)。?? ,????????? 0??? ,?????????????????? ,???????????????????? ,?????? 。????????????? 。??????? ,????????????????????????。?????????????? ,???????? 0? 1 ?? 。?????????? ,???????? ,????????? 。?? ,??????????????????????????? ,????????? ,?????????? ,???????。4　????????????????????????? ,?????????。????????????????????? 。??????? ,??????????????28????????。????????? ,?????????? 。????????????????? ,?????????????? ,????????。??????????? ,????????? ,?????????????。????????? ,?????? ,????????????????。???????????????? ,????????????????? 。??????????????? ,????????????????????????????? 。???????????? ,??????? ,???????????????? ,??????????????????。??????????????????? ,????????????????????????。??????? ,????????????? ,??????????????。?????????????????????。????????????? ,???????????????????。?　?　?　?:[ 1] 　M.Carl.Recent Research in the Field of Example-Based Machine Translation[ A] .CICLing 2001 , LNCS 2004.[ 2] 　W.John Hutchins.Machine Translation:a brief history.Concise history of the language sciences:from the Sumerians tothe cognitivists[ M] .Oxford:Pergamon Press , 1995.[ 3] 　Sumita , E.and H.Iida.Experiments and Prospects of Example-Based Machine Translation[ A] .Proceedings of 29thACL Meeting[ C] .Berkeley , 1991 , 185-192.[ 4] 　???.??????[ M] .???:?????????? , 2000.[ 5] 　K.Chidananda Gowda and E.Diday.Symbolic Clustering Using a New Similarity Measure [ J] .IEEE Transactions onSystems ,Man , and Cybernetic , 1992 , 22(2).[ 6] 　???.??????????????[ J] .???????? , 2002 , 38(7):75-76.[ 7] 　Federica Mandreoli , Riccardo Martoglia, and Paolo Tiberio.Searching Similar(Sub)Sentences for Example-Based Ma-chine Translation[ A] .In:Atti del Decimo Convegno Nazionale su Sistemi Evoluti per Basi di Dati(SEBD 2002), IsoladElba , Italy , 2002.[ 8] 　??? ,???.???????????????????[ J] .?????? , 2002 , 16(4):39-44.[ 9] 　??? , ?? ,???.??????????????????[ J] .?????? , 2003 , 17(1):60-64.[ 10] 　??? ,???.?????????????????[ A] .?????????????(ICCIP98)[ C] .??:??????? , 1998 , 458-465.29

A similarity measure method of Chinese sentence structures

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository