PACLIC 20 / Wuhan, China / 1-3 November, 200

Huang Chang-Ning

Li Mu

Lu Bao-Liang

Zhao Hai

English

DSpace at Waseda University

Effective Tag Set Selection in Chinese WordSegmentationvia Conditional Random Field ModelingHai Zhao 1,2? , Chang-Ning Huang 2 , Mu Li 2 , and Bao-Liang Lu 11 Department of Computer Science and Enginee ring, Shanghai Jiao Tong University, 1954 HuaShan Rd., Shanghai 200030, China{zhaohai,blu}@cs.sjtu.edu.cn,2 Microsoft Research Asia,49, Zhichun Road, Haidian District,Beijing, China, 100080cnhuang@msrchina.research.microsoft.com, muli@microsoft.comAbstract. This paper is concerned with Chinese word segmentation, which isregarded as a character based tagging problem under conditional random fieldframework. It is different in our method that we consider both feature templateselection and tag set selection, instead of feature template focused only methodin ex isting work. Thus, there comes an empirical comparison study of perfor-mance among different tag sets in this paper. We show that there is a significantperformance difference as different tag sets are selected. Based on the proposedmethod, our system gives the state-of-the-art p erformance.1 IntroductionChinese text is written without natural delimiters, so word segmentation is an essentialfirst step in Chinese language processing. In this aspect, Chinese is quite different fromEnglish in which sentences of words delimited by white spaces. Though it seems verysimple, Chinese word segmentation is not a trivial problem. Actually, it has been activearea of research in computational linguistics for almost 20 years and has drawn moreand more attention in the Chinese language processing community. To accomplish sucha task, various technologies are developed [1].To give a comprehensive comparison of Chinese segmentation on common test cor-pora, two International Chinese Word Segmentation Bakeoffs were held in 2003 and2005, and there were 12 and 23 participants respectively [4], [5].In all of proposed methods, character based tagging method [2] quickly rose in twoBakeoffs as a remarkable one with state-of-the-art performance. As reported in [5], theresults of Bakeoff-2005 shows a general trend to a decrease in error rates from 3.9%to 2.8% compared to the results of Bakeoff-2003. Especially, two participants, Ng andTseng, gave the best results in almost all test corpora [6], [7].We continue to improve CRF-based tagging method of Chinese word segmentationon the track of Ng and Tseng in this study. It is different in our method that we consider? This work was finished while the first auth or visited Microsoft Research Asia87both feature template selection and tag set selection, instead of feature template focusedonly methods in previous work. That is, feature template selection was the main workif it was not the unique one before, while tag set is empirically specified aforehand.There are two kinds of test schemes in Chinese word segmentation Bakeoff, openand closed test. In the open test participants are allowed to use training data and anyother linguistic resource including other training corpora, proprietary dictionaries andso forth. In the closed test only training data are allowed to used for the particularcorpus. No other data is allowed. In this study, we will limit our comparison in closedtest because additional linguistical resource often varies from system to system.The remainder of the paper is organized as follows. The next section is a simpleintroduction to conditional random field. Feature templates and tag sets are given inSection 3. In Section 4, our experimental results are demonstrated. We summarize ourcontribution in Section 5.2 Conditional Random FieldMaximum entropy tagger was used in early character-based tagging for Chinese wordsegmentation [2], [3], while we choose linear-chain CRF as our learning model in thisstudy. It can combine rich feature representation and probabilistic finite state model,too. In addition, it can avoid so-called ‘label-bias’ problem in some degree. Actually,such model was also proved to be very effective in many existing works [8].Conditional random field (CRF) is a statistical sequence modeling framework firstintroduced into language processing in [9]. Work by Peng et al. first used this frameworkfor Chinese word segmentation by treating it as a binary decision task, such that eachChinese character is labeled either as the beginning of a word or not.The probability assigned to a label sequence for a particular sequence of charactersby a CRF is given by the equation below:pλ(Y |W ) = 1Z(W )exp(∑t∈T∑kλkfk(yt−1, yt,W, t)) (1)where Y = {yi} is the label sequence for the sentence, W is the sequence of unseg-mented characters, Z(W ) is a normalization term, fk is a feature function, and t indexesinto characters in the label sequence.3 Tag Sets and Feature TemplatesCharacter based tagging method for Chinese segmentation, either based on maximumentropy or CRF, regards a segmentation procedure as tagging, which is described indetail in [10].The probability model and corresponding feature function is defined over the setH × T , where H is the set of possible contexts (or any predefined condition) and T isthe set of possible tags. Generally, a feature function can be defined as follows,f(h, t) ={1, if h = hi is satisfied and t = tj0, otherwise, (2)88where hi ∈ H and ti ∈ T .For convenience, features are generally organized by some groups, which used tobe called feature templates. For example, a bigram template C1 stands for the nextcharacter occurring in the corpus after each character .A feature template set we selected is shown in Table 1. We give an explanation tofeature template T−1T0T1 . Here, T−1 , T0 or T1 stands for predefined class. There arefour classes defined: numbers represent class 1, those characters whose meanings aredates represent class 2, English letters represent class 3, and other characters representclass 4. This feature template is improved from the corresponding one in [6]. Refer itfor more details.Table 1. Feature templatesType Feature FunctionUnigram C−1, C0, C1 The previous, current and next charac terBigram C−1C0, C0C1 The previous (next) character and curr ent characterC−1C1 The previous character and next characterPunctuation Pu(C0) Current character is a punctuationDate, Digital and Letter T−1T0T1 Types of previous, current and next ch aracterTable 2. Definitions of different tag setsTag set Tags Words in tagging2-tag (Peng/Tseng) B, E B, BE, BEE, ...4-tag (Xue/Ng) B, M, E, S S, BE, BME, BMME, ...5-tag B,B2,M,E, S S,BE,BB2E,BB2ME,BB2MME , ...6-tag B,B2, B3,M,E, S S,BE,BB2E,BB2B3E,BB2B3ME , ...As for tag set, there are two kinds of schemes are used to distinguish the characterposition in a word in the previous works. The detail can be referred to Table 2. NoticeXue and Ng use their 4-tag set in maximum entropy model. Two reported CRF methods,i.e., Peng and Tseng, used 2-tag set. Generally speaking, activated feature functions inpractice like (2) are determined by both feature template and tag set. In the existingwork, tag set is specified aforehand. Ng and Xue used a maximum entropy tagger withthe same 4-tag set, while Peng and Tseng used CRF. CRF is a sequence learner, andword segmentation is often regarded a binary decision procedure. Thus they chose 2-89tag set. We will see tag set selection is as important as feature template set selection inthe empirical comparison.To effectively perform tagging for those longer words, we extend 4-tag set of Ng/Xue.The tag ’ B2 ’ is added into 4-tag set to form 5-tag set, in which stands for the secondcharacter position in a Chinese word. Similarly, the tag ’ B3 ’ is added into 5-tag set toform 6-tag set, in which stands for the third character position in a Chinese word. Thisextension is based on our observation of average weighted word length distribution ineach corpus. We will give further discussions in Section 4.4 ExperimentsEight corpora are available from Bakeoff-2003 and 2005. We use all of them to performthe evaluation. A summary of these corpora is shown in Table 3.Table 3. Corpora statistics of Bakeoff-2003 and 2 005Provider Corpus Encoding #Training words #Test words OOV rateAcademia Sinica AS2003 Big5 5.8M 12K 0.022AS2005 Big5 5.45M 122K 0.043Hong Kong City University CityU2003 Big5 240K 35K 0.071CityU2005 Big5 1.46M 41K 0.074U. Penn Chinese Treebank CTB2003 GB 250K 40K 0.181Beijing University PKU2003 GB 1.1M 17K 0.069PKU2005 GB 1.1M 104K 0.058Microsoft Research Asia MSRA2005 GB 2.37M 107K 0.0264.1 Experimental Results under Different Tag SetsTo observe the trends of performance under different feature template sets and differenttag sets, we define another two feature template sets in addition to the feature templateset defined in Table 3. Their definitions are shown in Table 4. The experimental resultsof CityU2003 and PKU2005 are shown in Table 5. From the experimental results, wecan get a general trend of performance increasing from 2-tag to 6-tag set.As we have shown above, TMPT-01 is the template set which gives the best perfor-mance when it combines with 6-tag set. TMPT-02 is a pure n -gram template set. It canbe observed that TMPT-02 gives a substantial performance improvement when it workswith 6-tag set. On the contrary, the best set, TMPT-01 will lose its top place when itworks with 2-tag set.90Table 4. Four feature template setsID DescriptionTMPT-01 Defined in Table 1.TMPT-02 Add C−2, C2 , and remove Pu, T−1T0T1 in TMPT-01TMPT-03 Add C−2 in TMPT-01TMPT-04 Remove Pu and T−1T0T1 in TMPT-01Table 5. Experimental results of CityU2003 and PKU2005 under different feature template setsand tag setsFeature Set CityU2003 PKU20052-tag 4-tag 5-tag 6-tag 2-tag 4-tag 5-tag 6-tagTMPT-01 0.9320 0.9472 0.9478 0.9483 0.9505 0.9522 0.9531 0.9536TMPT-02 0.9302 0.9450 0.9461 0.9462 0.9461 0.9479 0.9499 0.9503TMPT-03 0.9324 0.9471 0.9464 0.9467 0.9493 0.9499 0.9523 0.9526There is another comparison between our system and Tseng’s: we propose 8 groupsof feature templates shown in Table 1, while there are 15 groups of selected featuretemplates in Tseng’s system. However, with a selected appropriate tag set, our systemgives its superior performance.4.2 Comparisons of Best Existing Results and Our ResultsThe comparison between our results and best existing results are shown in Table 6 3 .There are two types of existing results. One is the best F score of Bakeoff-2003 and2005 for each corpus under closed test. The other is the results of Tseng et al.. All ofour results are performed under TMPT-01 (or TMPT-04 4 ) and 6-tag set.3 The Third SIGHAN Chinese Language Processing Bakeoff has been held, the results will bepresented at the 5th SIGHAN Workshop, to be held at ACL-COLING 2006 in Sydney, Aus-tralia, July 22-23, 2006. We also participate this Bakeoff, and ou r system with the techniquespresented in th is paper won four highest and two third highest F measures in six Chinese wordsegmentation tracks. Our results on Bakeo ff-2006 appear in SIGHAN-2006 [11].4 Note that TMPT-04 is a feature template set only inclu ding n -gram ones. Researchers in C WSdid not make an agree on what P and T−1T0T1 are feature templates for closed test or not.Thus the results under TMPT-04 are demonstrated, too. We will see that our system gets state-of-the-art performance in either of feature te mplate set.91Table 6. Comparisons of best existing results and our results in the corpora of Bakeoff-2003 and2005Participant AS2003 CTB2003 CityU2003 PKU2003 AS2005 CityU2005 PKU2005 MSRA2005Peng 0.956 0.849 0.928 0.941Tseng 0.970 0.863 0.947 0.953 0.947 0.943 0.950 0.964Best of Bakeoff 0.961 0.881 0.940 0.951 0.952 0.943 0.95 0.964Ours/TMPT-01 0.973 0.873 0.948 0.956 0.954 0.956 0.954 0.974Ours/TMPT-04 0.973 0.872 0.947 0.956 0.953 0.948 0.952 0.9744.3 Determine Effective Tag SetsHow to select an effective tag set for current segmentation task is an interesting problem.Since our task is to segment sequence into various length words, it is natural for us toanalyze the word length distribution in a corpus.Average weighted word length distribution of eight concerned training corpora areshown in Table 7. The length of a Chinese word is measured by the number of includingcharacters. The numerical values in Table 7 are calculated through dividing the sum oflengths of words with specified length by the number of all words in the corpus. Wemay notice that average weighted word length of MSRA2005 and CTB2003 are thelongest. As for CityU2005, though it is not the shortest one, the average length of allwords which are longer than three-character is the shortest. This is what we are moreinterested in, since our tag set trends to extend the case with more tags.Notice 6-tag set can label a five-character word without repeating its tags, that is,’ BB2B3ME ’, we may take average weighted word length of those words whose char-acter lengths are larger than four as our experimental criteria to determine if 6-tag setshould be taken or not. For example, we may calculate a value through dividing the sumof lengths of those words whose lengths are larger than four characters by the numberof all words in corpus. The threshold is empirically set to 0.02 for our current task. Ifthe obtained value is larger than the threshold, then we adopt 6-tag set, otherwise, weadopt a tag set with five or less tags. Comparison results are shown in Table 9. Theexperimental results show that CityU2005 corpus with shortest average weighted wordlength among eight corpora gets its higher performance at 5-tag set instead of 6-tagset though the difference is slight, while MSRA2005 and CTB2003 corpora with thelongest word lengths win the most performance improvement from 5-tag to 6-tag set.We have not discussed the tag set with more than six tags until now. The reasonis still behind word length statistics. The distribution of words with different lengths ineight training corpora are shown in Table 8. We can see that the length of 99.89% wordsat least in all corpora are less than six characters. This also partially explains why 6-tagset works well in most cases. Actually, we tried tag sets with more than 6 tags, but wedid not obtain obvious improved performance in almost all cases.92Table 7. Average weighted word length distribution of eight training corporaWord Length AS2003 AS2005 CTB2003 CityU2003 CityU2005 PKU2003 PKU2005 MSRA2005Total 1.5458 1.5089 1.7016 1.6130 1.6275 1.6429 1.6455 1.7101≥ 2 1.001 0.9378 1.2649 1.1190 1.1586 1.1708 1.1728 1.2401≥ 3 0.2135 0.1804 0.3211 0.2648 0.2479 0.2692 0.2730 0.3619≥ 4 0.0747 0.0730 0.1195 0.0887 0.0688 0.1208 0.1244 0.2193≥ 5 0.0320 0.0334 0.0732 0.0252 0.0150 0.0390 0.0423 0.1223≥ 6 0.0228 0.0241 0.0351 0.0133 0.0072 0.0105 0.0142 0.0776Table 8. The distribution of words with different length s in eight training corporaWord Length AS2003 AS2005 CTB2003 CityU2003 CityU2005 PKU2003 PKU2005 MSRA2005≤ 5 0.9973 0.9974 0.9950 0.9981 0.9990 0.9985 0.9983 0.98996 0.0008 0.0007 0.0024 0.0010 0.0005 0.0007 0.0007 0.0037≥ 7 0.0019 0.0019 0.0026 0.0009 0.0006 0.0008 0.0010 0.0063Table 9. Relation between tag set and average weighted word length under feature template setTMPT-01Participant CTB2003 CityU2003 CityU2005 PKU2003 PKU2005 MSRA20056-tag 0.8727 0.9483 0.9563 0.9559 0.9536 0.97375-tag 0.8715 0.9478 0.9567 0.9554 0.9531 0.9724Difference 0.0012 0.0005 -0.0004 0.0005 0.0005 0.0013average weighted word length 0.0732 0.0252 0.0150 0.0390 0.0423 0.1223(Length ≥ 5)935 ConclusionIn this work, we have shown an appropriate tag set can give a substantial performanceimprovement of Chinese word segmentation for character based tagging method underCRF framework. Furthermore, we propose that average weighted word length distribu-tion of the corpus can be taken as the criteria to choose tag set. Based on the proposedmethod, our system obtains state-of-the-art performance in all corpora of Bakeoff-2003and 2005.References1. Jianfeng Gao, Mu Li, Andi Wu and Chang-Ning Huan g. 2005 . Chinese Word Segmenta-tion and Named Entity Recognition: A Pragmatic Approach. Computational Linguistics , Vol.31(4): 531-574.2. Nianwen Xue. 2003. Chinese Word Segmentation as Character Tagging. ComputationalLinguistics and Chinese Language Processing , Vol. 8(1): 29-48.3. Nianwen Xue and Libin Shen. 2003. Chinese Word Segmentation as LMR Tagging. In Pro-ceedings of the Second SIGHAN Workshop on Chinese Language Processing, in conjunctionwith ACL’03 , 176-179. Sapporo, Japan.4. Richard Spro at and Thomas Emerson. 2003. The First International Chinese Word Seg-mentation Bakeoff. Proceedings of the Second SIGHAN Workshop on Chinese LanguageProcessing , 133-143. Sapporo, Japan.5. Thomas Emerson. 2005. The Second International Chinese Word Segmentation Bakeoff.Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing , 123-133.Jeju Island, Korea.6. Jin Kiat Low, Hwee Tou Ng and Wenyuan Guo 2005 . A Maximum Entropy Approach toChinese Word Segmentation. Proceedings of the Fourth SIGHAN Workshop on Chinese Lan-guage Processing , 161-164. Jeju Island, Korea.7. Huihsin Tseng, Pichuan Chang, Galen Andrew, Daniel Jurafsky, Christopher Manning. 2005.A Conditional Ra ndom Field Word Segmenter for Sighan Bakeoff 2005. Proceedings of theFourth SIGHAN Workshop on Chinese Language Processing, 168-171. Jeju Island, Korea.8. Fuchun Peng, Fangfang Feng and Andrew McCallum. 2004. Chinese Segmentation and NewWord D etection Using Co nditional Random Fields. In COLING 2004 , 562- 568. Geneva,Switzerland, August 23-27, 2004.9. John Lafferty, A. McCallum and F. Pereira. 2001. Conditional Random Field: Probabilis-tic Models for Segmenting and Labeling Sequence Data. In Proceedings of the EighteenthInternational Conference on Machine Learning, 282-289. June 28-July 01, 200 1.10. Adwait Ratnaparkhi. 1996. A Maximum Entropy Part-of-speech Tagger. In Proceedings ofthe Empirical Method in Natural Language Processing Conference, 133-142. University ofPennsylvania.11. Hai Zhao, Chang-Ning Huang and Mu Li. 2006. An Improved Chinese Word SegmentationSystem with Conditional Random Field. Proceedings of the Fifth SIGHAN Workshop onChinese Language Processing , 108-117. Sidney,Australia.94

Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field Modeling

PACLIC 20 / Wuhan, China / 1-3 November, 2006conference pape

Zhao, Hai

50208

Huang, Chang-Ning

50209

Li, Mu

50210

Lu, Bao-Liang

50211

Waseda University Repository

https://waseda.repo.nii.ac.jp/?action=repository_action_common_download&item_id=28916&item_no=1&attribute_id=101&file_no=1

Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field Modeling

Abstract

Similar works

Full text

Available Versions

DSpace at Waseda University

Waseda University Repository