Cheung Lawrence

Sun Maosong

Tsou Benjamin K.

English

Cheung, Lawrence

49652

Tsou, Benjamin K.

49653

Sun, Maosong

49654

Waseda University Repository

Identification of Chinese Personal Names in Unrestricted Texts.Lawrence CHEUNG, Benjamin K. TSOULanguage Information Sciences Research CtrTat Chee Avenue, Kowloon TongHong Kong{rlylc, rlbtsou} @cityu.edu.hkMaosong SUNNational AI Lab.,Tsinghua University, Beijing,Chinalkc-dcs@tsinghua.edu.cnAbstractAutomatic identification of Chinese personal names in unrestricted texts is a key task in Chineseword segmentation, and can affect other NLP tasks such as word segmentation and informationretrieval, if it is not properly addressed. This paper (1) demonstrates the problems of Chinesepersonal name identification in some IT applications, (2) analyzes the structure of Chinesepersonal names, and (3) further presents the relevant processing strategies. The geographicaldifferences of Chinese personal names between Beijing and Hong Kong are highlighted at theend. It shows that variation in names across different Chinese communities constitutes a criticalfactor in designing Chinese personal name identification algorithm.Keywords: Chinese personal name identification, Chinese word segmentation, Chinese ITapplications, Chinese linguistic differences1	 IntroductionAutomatic identification of Chinese personal names in unrestricted texts plays a key role in ChineseNLP tasks such as word segmentation. Personal name identification in Chinese involves many differentissues that are not found in similar tasks in English. Despite extensive research in the issue in recentyears (Wang et al. 1992, Song et al. 1993, Sun et al. 1995), many NLP applications still suffer from theweakness of current available performance.The paper has four sections: Section 1 illustrates some IT applications in which Chinese personalname identification plays important roles, to promote an appreciation of the significance of the research.Section 2 introduces the structure of Chinese personal names, and Section 3 discusses the relevantprocessing strategies. Lastly, we will highlight the significant differences of Chinese personal namesbetween Beijing and Hong Kong in Section 4, showing the added difficulty caused by variations indifferent Chinese communities.2	 The Role of Automatic Identification of Personal Names in IT Applications2.1 Information Retrieval (IR)* This research is supported in part by the National Basic Research Scheme of China, under grant no.G199803050 and by Language Information Sciences Research Centre, City University of Hong Kong.28IR systems, including search engine on WWW, retrieval system of digital library, text mining etc.,search and retrieve the requested information from large bodies of data stored in electronic form. Withthe advent of the Internet, search engine techniques have become most widely influential. However; theeffectiveness of current systems is not satisfactory especially for retrieving Chinese web sites/pages. Wehave examined how personal names interfere with the correct retrieval of relevant results in Sinai, apopular Chinese search engine, and Google [Big5 Chinese] 2. For instance, suppose we want to get someinformation about "EN:" (lieutenant general) from WWW, and enter that word as a query to. Sina, thensome of the "relevant" web pages will be returned, as in (1)—(4):(1)Mitigt4111(2)AiktiStli. .f r1A-11-SAMP ITIXti (0.10(3)ISIRMVVIIMR40fig143/C.P32a10N(4) irtiFIVVE	 UM4INITIgitPirlArt -** °(Sina)(Sina)(Google [Big5 Chinese])(Google [Big5 Chinese])These webpages are actually irrelevant to "lieutenant general" although the string "414" is embedded.The string "4:1" in the two sentences cannot be analyzed as the word meaning "lieutenant general". In(1) and (2), the character "4" is the given name, and 1:" an adverb by itself meaning "will" forreferring to the future. In (3) and (4), "11:1:" forms the two-character given name. The misinterpretationwould have been avoided if the search engine could detect the presence of proper names and segment thewords accordingly.2.2 Text-to-Speech Conversion (TTS)The text-to-speech conversion system accepts textual input and generates the corresponding speechsignal. It finds applications in mail reading over the phone, generating spoken prompts in voice responsesystems, human-machine interface and so on. Lucent Technologies hosts a web site called Bell LabsMandarin Text-to-Speech Synthesis 3 which allows the user to input text and outputs the correspondingspeech. We tested sentences containing characters that are pronounced differently when it functions as asurname and as a character with other meaning. Any incorrect pronunciation will suggest that the systemfails to differentiate personal names from other content words. Consider (5) first: si(5) Iseirerm A.4 Trii	 itf4	 [YAM 0zha	 zha (cha)"My boss Zha Jin-Tai did not agree to the opinion of his younger brother, Mi\Zha Jiaripuo."In Mandarin, "A" can be pronounced as cha (a mono-syllabic word meaning "to check") or zha(surname). In (3), both occurrences of "If" serve as a surname. The synthesized ,ispeech correctlypronounces the first occurrence as zha, but mis-pronounces the second one as cha, indicating that theChinese name "A*" is correctly identified whereas "*4 " is not (As can be seen in the Section 3,the identification of the former one is easier that that of the latter). In this case, the language processingmodule "under-recognizes" Chinese names.The other case is Chinese name being "over-recognized". Input (6) to Bell Labs TTS. The character"X" acquires different tones in different contexts. It is pronounced as hual (a morpheme meaningChina) or hua4 (surname). The character " " is another potential trap. It is pronounced as ceng2 (anadverb meaning "once") or zeng4 (surname).1 Sina Search Engine website: http://www.sina.com.cn2 Google [Big5 Chinese] website : http://www.google.com/intl/zh-TW/3 Bell Lab TTS website: http://www.bell-labs.Com/project/tts/rnandarin-gb.html29(6) ‘2.:4 1:11*ARJ#111 TfliAPP,hua4 zengl hua2"Hua Guo-Feng is the former premier of the People's Republic of China."The system correctly differentiates the two occurrences of "*" and pronounces the characters correctly.However, it misinterprets " {-1" as a surname and pronounces " " as zengl . Apparently, theconversion system has not been able to identify personal name reliably and made errors in generating thecorrect speech output.2.3 Machine Translation (MT) SystemMT application is another important NLP application. The rise of the Internet gives rise to a greatdemand for translating web pages on the fly from a foreign language into the native language. AlthoughMT systems are far from perfect, many free web-based MT systems have already been capable of doingreasonably good translation. We select the popular Transtar website4 in Mainland China for testing. Itoffers Chinese-to-English Machine Translation. The MT system translates (7a) and (8a) as (7b) and (8b)accordingly.(7) a. R, Z-51§5/1\*Pdailgr1'mrifb. "I see that Deng Xiao-Ping greets with Jiang Zemin."(8) a. IMII5	 °b. "I see week star Chi open together study friend greet."At first glance, (7a) and (8a) are identical in syntactic and semantic structures, except the personalnames. However, while (7b) is a good translation, (8b) is completely incomprehensible. An examinationof (7b) and (8b) shows that the major difference lies in the system's failure to identify the two personalnames in (8). It is likely that the two paramount political figures' names in (7a) are registered in thesystem's lexicon and the information can be called up in processing. It makes it possible to parse theChinese string and to generate the English translation. In contrast, the names' in (6b) are relatively lesswell-known in Mainland China and are not in the system' s lexicon, resulting in the brute-forcetranslation of "ji9" to "week", "*" to "star", "a" to "Chi"("," is a bound morpheme and the systemsimply output its ronianization of the character), "3K" to "open", "s" to "study", "SC to "friend"respectively.3	 Processing Chinese Personal Names: Challenges and Strategy3.1 Structure Of Chinese Personal NamesThe basic structure of modem Chinese personal names is largely similar across different communities.The frequent length is 2 or 3 characters. The maximum can be as long as 6 characters. Table 1 shows thepossible structures of Chinese personal name. Chinese personal name begins with the surname whichcan be one-character (as in (a) and (b)) or two-character (as in (c) or (d)). It is followed by the givenname which can be one-character (as in (a) and (c)) or two-character (as in (b) or (d)). A unique structureexists in some Chinese communities such as Hong Kong: the name of a married female may be precededby her husband's surname, as in (e) and (f). Such usage occurs in formal occasions or writing of formalregister.4 Transtar Website: http://www.transtar.com.cn/5 They are famous movie star and singer in Hong Kong.30Full Name Husband'sSurnameSurname GivenNameLengthH1 (H2) S 1 (S2) + G1 (G2)a.	 	 1111= 111=b. 'N. /J\* /..1\ / • 3c. r--,-A4A; il-A 0; -v.-f-gt.,'d. * - 111-14ffl 1-11-1e. 13*. P* 4f. f-i-AX* - 1104ffl 7.A RI 6Table 1. Possible variations of Chinese personal names3.2 ChallengesSeveral characteristics make it difficult for computers to recognize Chinese personal names. Unlikecapitalization of personal names in English, there is no explicit marking for personal names in Chinese.Other problems include: (1) Chinese texts do not have explicit word boundaries; (2) the character set forsurname and given name are strictly a subset of Chinese character set that forms common Chinese words;(3) some characters in personal names may be mono-syllabic words; (4) some multi-syllabic words canbe involved in Chinese personal names starting at every possible position. All of these factors introducea lot of potential ambiguities in Chinese personal name identification.The situation becomes even more complicated regarding the fact that some polysyllabic commonwords are also possible in names, e.g. "OBJJ" ("*JJ"= dynasty), "%WU" ("Mir = victory) and44	("	 = serious(ly)). In these cases, the algorithm may easily mistake names as commonwords and produce irrecoverable errors. This is illustrated in (9) regarding "= 10'1 :7E9(9) ***rrd=t.41 INffi73RYMAI , 	  it1JIL),E3tE(Somebody)... meanwhile, seriously pointed out the mistake of Zhang Yi, hoping he could correct it.Although "Am" is interpreted as a common word meaning "serious", it is possibly a personal name. Soanother interpretation seems to be reasonable if larger context (perhaps the structure of the wholesentence) is referred to:(10) • Meanwhile, Yan Su pointed out the mistake of Zhang Yi, hoping he could correct it.3.3 Basic StrategiesChinese personal names are so elusive that locating them in running texts usually relies on acombination of techniques and knowledge to achieve satisfactory recall and precision rate. In thefollowing, several identification strategies will be outlined. They are broadly divided into two types,namely, inherent characteristics of personal names and contextual information of texts.313.3.1 Inherent Characteristics of Personal NamesProbabilityCertain combinations of characters are more likely to form a name. Supported by statistics derived froma large scale Chinese personal name database, the identification procedure can guess name candidatesby figuring out the probability of any character string being a name.log(prob( gt-X))= -5.02 > > log(prob(M))= -6.20From the calculation, it is possible to predict that the combination "po" is more likely to be a namethan the combination "M".ConstructionFor example, reduplication of characters can be found in Chinese names such as "and "WF". The program may alert the identification procedure when reduplication patterns areencountered.3.3.2 Contextual Information of TextsContext can often provide important cues to locating Chinese personal names. Title is a good indicatorthat the characters immediately before or after it is a potential personal name, e.g. 3AV-1.4-t. (Mr.), .T14±-(Dr.) and tilV44(Premier). Frequently used patterns or syntactic structures may also beuseful for identifying names, for example,(11) tk <name> <titlel> {title2}e.g. iJ,<' f±*>	 RoicA4.1. f-tAIN(12) { {organization} {title} } <name>e.g. { on} {Iffg} <(_v.-TR>National AI Lab at Tsinghua University (Sun et al. 1997) developed a Chinese word segmentationprototype system that integrates all the above strategies for name identification. It achieves 95.0% recalland 87.6% precision in open test for Chinese personal name identification.4	 Geographical Differences of Chinese Personal NamesGenerally speaking, different Chinese communities may share similar convention of naming, but we dofind divergence in specific personal name structure and choice of characters. In this section, we willcompare personal name data drawn from Beijing and Hong Kong. The statistical findings not only bearsociolinguistic interest about divergence of difference Chinese communities but also present essentialcues to accurate personal name identification in texts from different Chinese communities. Theknowledge derived can be applied to devise better techniques and customize identification algorithmsfor different Chinese texts.4.1 DataTwo databases of Chinese personal names are collected from Beijing and Hong Kong. The Beijingdatabase is drawn from a name list of a county in Beijing. The database that contains 125,033 names isrepresentative of names in Mainland China because the county population is composed of migrants from32different provinces of China. The Hong Kong database contains 11,358 names. They are student andstaff names taken from Registrar's Office, City University of Hong Kong.4.2 Findings4.2.1 Distribution of SurnamesThe Chinese surname consists of one or two characters. Single-character surnames are predominant,accounting for over 99% in both databases.Beijing (%) HK (%)Single-Char. Surname 99.92 99.56Double-Char. Surname 0.08 0.44Table 2. Distribution of single- and double-character surnamesThe frequency of the surnames is ranked. Table 3 shows that the ranking of the top ten surnames in thetwo databases are quite different. The most striking difference is that the most frequent surname .1.(Wang) in Beijing is ranked as 14th in the Hong Kong.BeijingSurnameHong Kong%	 I Cum. %Rank Surname % Cum. % Rank1 E 9.12 9.12 1 PV 10.16 10.162 8.33 17.45 2 * 6.71 16.873 7.92 25.37 3 5.87 22.744 lij 6.47 31.84 4 fen, 4.57 27.325 N 3.17 35.01 5 4 4.19 31.516 tfl 3.15 38.16 6 3.64 35.157 tg 2.99 41.15 7 VI 3.03 38.198 I* 2.03 43.18 8 2.96 41.159 % 1.71 44.89 9 {J 2.81 43.9610 1.62 46.51  10 *5 2.09 46.05Table 3. Ten most frequent single-character surnames in Beijing and Hong Kong(Surnames found on both columns are shaded.)4.2.2 Distribution of Given NamesThere is a divergence between Beijing and Hong Kong in the preference for single- anddouble-character given names. Single-character names account for 29% of the Beijing database. Incontrast, single-character given names are not common in Hong Kong. They cover only 2% of the data.Beijing (%) HK (%)Single-char. 29.07 2.13Double-char. 70.93 97.87Table 4. Distribution of single- and double-character given names33The preference for characters in given names is rather different in the two databases as well. Table 5 and6 list the most common characters found in given names. When the columns for Beijing and Hong Kongare compared, only one or two characters overlap. Apparently, the two Chinese communities displayconsiderable difference in the choice of characters in naming.Beijin HKRank G1 %	 I Cum. % Rank G1 % Cum. %gl 3.18 3.18 1 7, 3.76 3.762 3 3.08 6.26 2 14 3.66 7.423 2.88 9.14 3 ,c,,, 3.54 10.974 ER 2.57 11.71 4 * 2.78 13.755 2.28 13.99 5 2.22 15.976 A 2.19 16.17 6 2.13 18.107 ,	 . 1.86 18.03 7 t),ji 1.99 20.098 /1\ 1.78 19.81 8 3 1.95 22.049 fl 1.65 21.46 *I 1.93 23.9710 4 1.35 22.80 10 .-4 1.88 25.85Table 5. Ten most frequent first characters (G1) of double-character given names(Surnames found on both columns are shaded.)Beijin& HKRank G2 %	 I Cum. %  Rank G2 % Cum. %1 :-: 3.63 3.63 1 1A 3.07 3.072 K  3.37 7.00 2 2.28 5.343 gi  2.10  9.09 3 2.23 7.574 * 1.91 11.01 NI 2.15 9.725 1.83 12.84 5_2.08 11.806 1.66 14.50 6 ft 1.88 13.687 ''''' 1.56 16.06 7 um1 1.74 15.428 4 1.50 17.55 8 a 1.58 17.009 --lh- 1.32 18.87 9 IX 1.57 18.5710 * 1.25 20.13 10 --- 1.55 20.12Table 6. Ten most frequent second characters (G2) of double-character given names(Surnames found on both columns are shaded.)4.2.3 Co-occurrence of Characters in Double-character NamesOur data reveals that some characters tend to co-occur to form a double-character given name. Theinformation is useful for automatic identification. Given G1 = A and 1* (the two most common G1characters from Hong Kong), there is about 30% of chance that the given name is one of thecombinations in la-1 e and 2a-2e (Table 7).34Namecombination% Cum. % Namecombination% Cum. %la Ni	 t-l‘ 11.2 11.2	 '	 2a 14.+3 6.4 6.4lb-A-FIA 6.5 17.7	 2b 1-F 5.7 12.1lc ri+- 4.6 22.3	 2c 4-+tiii 5.7 17.7ld A+ M- 4.3 26.6	 2d {4. + 5.4 23.2le A+M 3.6 30.1	 2e it + FIA 5.2 28.3Table 7. Five most frequent combinations given Gi = A or M.5	 ConclusionThis paper discusses the challenges and basic strategies of personal name identification in unrestrictedChinese texts. The reviewed IT applications are still not reliable in processing personal names. Thefailure proliferates into further errors in other NLP tasks such as speech generation and parsing. Thedistributional and structural differences between names in Beijing and Hong Kong are also presented.The findings imply that personal name identification systems, especially those that arestatistically-based, have to take into account the differences of personal names among various Chinesecommunities in order to refine the precision and recall.AcknowledgementWe would like to thank Prof Rou SONG for his kindness in providing us with the Beijing personal namedatabase, and Registrar's Office, City University of Hong Kong, for their contribution to our HongKong personal name database.ReferencesSong, R, H. Zhu, W. Pan and Z. Yin. (1993). Automatic Recognition of Person Names Based on Corpusand Rule Base. Research and Application of Computational Linguistics, Beijing Language InstitutePress, Beijing, China.Sun M., C. Huang, H. Gao and J. Fang. (1995). Identifying Chinese Names in Unrestricted Texts.Journal of Chinese Information Processing, 9 (2), Beijing, China.Sun M., D. Shen, and C. Huang. (1997). "CSeg&Tag1.0: A Practical Word Segmenter and POS Taggerfor Chinese Texts," Proceedings of the 5th Intl Conference on Applied Natural LanguageProcessing, Washington DC, USA.Wang, L.J., W. C. Li and C. H. Chang. (1992). Recognizing Unregistered Names for Mandarin WordIdentification, Proceedings of COLING-92, Nantes, France.35

Identification of Chinese Personal Names in Unrestricted Texts

DSpace at Waseda University

https://waseda.repo.nii.ac.jp/?action=repository_action_common_download&item_id=28744&item_no=1&attribute_id=101&file_no=1

Identification of Chinese Personal Names in Unrestricted Texts

Abstract

Similar works

Full text

Available Versions

Waseda University Repository

DSpace at Waseda University