Choi Byung-Jin

Choi Key-Sun

Lee Jae-Sung

Lee Woon-Jae

English

Choi, Byung-Jin

49202

Lee, Jae-Sung

49203

Lee, Woon-Jae

49204

Choi, Key-Sun

49205

Waseda University Repository

Language, Information and Computation(PACLIC 11), 1996, 129-136A Logical Structure for the Construction of Machine ReadableDictionariesByung Jin Choi** , Jae Sung Lee* , Woon Jae Lee* and Key Sun Choi**Department of Computer Science, KAIST**Center for Artificial Intelligence Research, KAISTAbstractDuring the last 10 years, there have been many efforts in some areas of NaturalLanguage Processing to encode the normal text or documents into machine readable form.If we encode written data using a canonical form which can be recognized by a computer,we can extract needed information and process and utilize it for another purposes. Fromthis point of view, we present an account of the encoding of a printed dictionary. Theconstruction of a lexicon is very time-consuming and expensive work and the applicationof the lexicon is restricted. In this paper, we describe a logical structure for Korean printeddictionaries as a general lexical representation based on SDML, which can be transformedinto another representation for different application requirements.1. IntroductionWith the rapid development of the computer technology, there are many effortsin some area of Natural Language Processing to convert normal text or documents intomachine readable form. If we convert written data into a canonical form which canbe recognized by computer, we can extract needed information and process and utilizeit for another purpose.In order to do this, we need to encode documents in a markup system. For thelast 10 years, there have been many approaches to develop a mark-up system in manyNLP projects. A Mark-up system is a set of instruction, with which a document or textcan be stored in a formalized form. In other words, we can build complex datastructures with a limited number of tags of this mark-up system.In the development of mark-up systems, standardization problems have been anissue. At the beginning of the 1980s, some researchers tried to develop encodingsystem and at last SGML (Standardized Generalized Markup Language) and TEI (TextEncoding Initiative) are emerged. With the help of SGML or TEI many documentsor texts are structured as DTD (Document Type Definition) of SGML or TEI (Bryan1988, Ide et a1.1995, Sperberg-McQueen et al. 1994).In this paper, we present an account of the encoding of printed dictionary. Theconstruction of a lexicon for a limited application domain is both time-consuming andexpensive. When we define and implement a lexicon only for some natural languageapplications, we cannot re-use this lexicon for another applications. To avoid thisproblem, we can encode the lexical information in a standard format. If we have thelexical information in a standard format, we can transform into the machine readableform or vice versa.If a lexical information in one system is compatible with another application, wecan reuse the existing standard lexicon and save our efforts to build a lexicon. Such a129reusability of existing lexical information is the main focus of this paper. Calzolari hasdistinguished two kinds of reusability (Kugler 1995). One is transforming alreadyexisting lexical resources into a different format, typically transforming printeddictionaries into a machine readable or machine tractable form (Amsler 1988, Alshawi1989). The other is exploiting already existing lexical resources for different theoriesand -typically - for different applications (Briscoe et al. 1993, Hajicovi & Rosen 1994).Such an idea is realised in this work.In order to maintain the reusablity of lexical information, we have developed theStandard Dictionary Markup Language (SDML). With SDML we can define thelogical structure of lexicon and store lexical information independent of specificapplications, data structures and theories.With the help of decoding program we can import the lexical information fromprinted dictionaries, text corpora, and some lexical databases into the standarddictionary. This information from the standard dictionary is then available for variousnatural language applications. In other countries there have been many such projectsduring the last 10 years. But we have no representative work on machine readabledictionary or electronic dictionary yet, except for the work of Kang (1996) aboutencoding Korean dictionaries. Kang adopted the TEI scheme for the encoding ofKorean dictionary entries. We have tried an SDML-based encoding, which is adaptedto dictionary entries. Because of its simplicity, it is easy to create a lexicon structurethat can be applied across various possible dictionaries.2. Standard Dictionary Markup Language (SDML)SDML is used to define the various dictionary formats. Once the dictionariesand text formats are defined using SDML, they can be easily for the standardizationand interchange of the information.In a SDML definition we have three parts, header, front (pre-definition) andentry group. <Header> is used for dictionary information such as dictionary name,dictionary version etc. In <Front>, the attributes and values which are needed forinternal structure are defined. The <entry group> comprises the lexical information ofdictionaries. Entries are repeated, in which the headword (lexical entry) is described.<sd><sdHeader> [header information] </sdHeader><group><entry></entry><entry><wname> [headword] </wname><body> [standard dictionary elements] </body><wname> [headword] </wname><body> [standard dictionary elements] </body></entry>[Repetition of entry]</group></sd>The elements which occur in <body> are regarded as standard dictionary format.130It can have only one field as a default value, or also have complex tree structures. Auser can define the logical structure of lexicon in <body> and this definition isoverriding and is overwritten the default definition. For the definition in SDML, itseems to be similar to SGML, we have some restrictions in using the rules.1) In the content model, we use the occurrence indicators, "1", `+' and "?', andas connectors `,' and `I'. The connector `8E.' is redundant for defining thedictionary format.2) For reasons of compatibility, predefined tags should be used. (in the upgradeversion, the more standard tags will be defined.)The default definition for <body> is following:<!ELEMENT body - 	 (#PCDATA)* >In order to build a new structure for <body>, we only need to write the definition forneeded elements. As an example we can change the structure of <body> with theoverwriting definition of <pos> and <del>.<!DOCTYPE	 sd	 SYSTEM "sdml.dtd" [<!ELEMENT	 body - o	 (pos, def) ><!ELEMENT pos - o	 (#PCDATA) ><!ELEMENT def - o	 (#PCDATA) >]><sd></sd>3. Representation of lexical entries in SDMLThe first thing that we take into account is how to encode lexical information inorder to extract relevant information efficiently. Each dictionary has a differentstructure (macro structure) and each dictionary entry varies extremely in its structure(micro structure) both within and among dictionaries. (Ide et al. 1995)Therefore, it is very difficult to find a general structural description which canbe applied across all possible dictionaries.In dictionary compiling, the macro structure is associated with the order oflexical entries. How to find a word in dictionary (e.g. according to alphabet orconcept) is the problem of constructing macro structure. How the information ofheadword is organized is the problem of micro structure design. In Korean dictionaries,the macro structure may vary a lot, therefore only the micro structure of lexical entriesis described.There are many kinds of Korean dictionaries (Samsung, Minjung, Kumsung,Donga etc.). We have compared these dictionaries, analysed each lexical entry andspecified the atomic elements which appear in a dictionary entry. Many of them are131different in their internal structure but there are also relevant common elements. Thecommon lexical information is following:headword (orthography), homograph number, pronunciation, part of speech,inflection class, etymology, terminology classification, usage information,definition, cross-reference, related headword ( antonym, synonym, hyponymetc. ), example, source, saying, idiom, derivationThis information is hierarchically organized. As a top level there is ainformation of word form which comprises pronunciation, orthography, and foreignorthography. At the same level as the lexical entries, we have grammatical information,semantic information, and derivational morphological information. All these aresubclassified and contain specified information in their respective subhierarchies. Thisinternal structure can be displayed as a tree:Lexical entryHeadword+ —orthography—headword number (homograph number)i—pronunciation+_foreign orthography+chinese notaionroman notation_grammatical information- part of speech- inflection form- etymologykorean etymologyforeign etymology- subcategorization_semantic information+- polysemy number- terminology area+- explanation- usage information*usage exampleusage example source- saying information*sayingsaying explanation- idiom information*idiom exampleidiom explanation- cross-reference*alternative headwordrelation of alternative headword__derivation- extended wordform- part of speech132This graphic representation is encoded in SDML in the following way:DTD definition<!DOCTYPE WDDIC [<!ELEMENT WDDIC - - (le)+ ><!ELEMENT le - - (hw , hwn , (fn I pr)+ , (gi , si+)+ ,wfext*) ><!ELEMENT hw - - (#PCDATA) ><!ELEMENT hwn - 0 (#PCDATA) ><!ELEMENT fn - - (ch I rom) ><!ELEMENT ch - 0 (#PCDATA) ><!ELEMENT rom - 0 (#PCDATA) ><!ELEMENT pr - 0 (#PCDATA) ><!ELEMENT gi - - (pos , fl? , etym?, subc?) ><!ELEMENT pos - 0 (#PCDATA) ><!ELEMENT fl - 0 (#PCDATA) ><!ELEMENT etym - - (ketm fetm) ><!ELEMENT ketm - 0 (#PCDATA) ><!ELEMENT fetm - 0 (#PCDATA) ><!ELEMENT subc - 0(#PCDATA) ><!ELEMENT si - - (num , term+ , expl , usage*,saying* , idiom* , althw*) ><ELEMENT num - 0 (#PCDATA) ><!ELEMENT term - 0 (#PCDATA) ><!ELEMENT expl - 0 (#PCDATA) ><ELEMENT usage - - (us_ex , us_source?) ><!ELEMENT us_ex - 0 (#PCDATA) ><!ELEMENT us_source - 0 (#PCDATA) ><ELEMENT saying - - (sng , sng ex?) ><!ELEMENT sng - 0 (#PCDATA) ><!ELEMENT sng ex - 0 (#PCDATA) ><!ELEMENT idiom - - (idm , idm ex) ><!ELEMENT idm 0 (#PCDATA) ><!ELEMENT idm ex - 0 (#PCDATA) ><!ELEMENT alt hw - (hw , alt_hw_rel) ><!ELEMENT althwrel - 0 (#PCDATA) ><!ELEMENT wf ext - - (hw , pos)+ >1>ExplanationDictionarylexical entryheadwordhomograph numberforeign notationchinese notationroman notationpronunciationgrammatical informationpart of speechinflection formetymologykorean etymologyforeign etymologysubcategorizationsemantic informationpolysemy numberterminology areaexplanationusage informationusage exampleusage example sourcesaying informationsaying examplesaying explanationidiom informationidiom exampleidiom explanationalternative headword(cross-reference)relation of alternative headwordextended wordform: derivation4. ConclusionIn this paper, we tried to describe the structure of a Korean standard dictionary,which is first encoded in a standard format and can then be reused for variousapplications. On the basis of SDML, we could define a DTD for the logical structureof standard dictionary. If a standard dictionary can be organized this way, we canextract many useful linguistic information and utilize many kind of dictionaries suchas synonym dictionary, antonym dictionary, idiom dictionary, example dictionary, anddictionaries for morphological analysis etc. Furthermore we can process these133dictionaries and use them for NLP systems, e.g. machine translation, automaticlanguage analysis, and information retrieval etc. In this way we can save time andeffort for building other lexicons for specific purpose. We have implemented thisstandard dictionary in TDMS (Text and Database Management System). In the future,we will concentrate on the implementation of structured information retrieval forextracting arbitrary information from the lexical database. Furthermore, the integrationof non-monotonic logic is desirable for the lexical organization.AcknowledgementsThe research described here was undertaken as a part of the project 'Korea Information BaseSystem' by the support of Ministry of Science & Technology and Ministry of Culture & Sportsin Korea and the project 'Multimedia Hangeul Engineering' supported by Samsung Co..ReferencesAlshawi, Hiyan. 1989. Processing Dictionary Definitions with Phrasal PatternHierarchies. In B. Boguraev and E. Briscoe, eds., Computational Lexicographyfor Natural Language Processing, 153-170. London and New York: Longman.Amsler, Robert A., and W. Tompa. 1988. An SGML-based Standard for EnglishMonolingual Dictionaries. In Proceedings of the 4th Annual Conference of theUW Centre for the New Oxford English Dictionary, 61-80. Ontario: Waterloo.Boguraev, Bran. 1991. Special Issue on Computational Lexicons. InternationalJournal of Computational Lexicography 4.Boguraev, Bran. 1994. Machine-readable dictionaries and computational linguisticsresearch. In A. Zampolli, N. Calzolari, and M. Palmer, eds., LinguisticaComputationale, Vol. LY.X Current Issues in Computational Linguistics: inHonor of Don Walker, 119-154. Dordrecht: Kluwer Academic Pub.Boguraev, Bran., and T. Briscoe (eds.). 1989. Computational Lexicography forNatural Language Processing. London and New York: Longman.Briscoe, Ted, V. de Paiva, and A. Copestake (eds.). 1993. Inheritance, Defaults, andthe Lexicon. Cambridge: Cambridge University Press.Bryan, Martin. 1988. SGML: An Author's Guide to the Standard Generalized MarkupLanguage. Addison-WesleyCopestake, Ann. 1990. An Approach to Building the hierarchical Element of a LexicalKnowledge Base from a Machine Readable Dictionary. In Proceedings of theFirst International Workshop on Inheritance in Natural Language Processing,19-29. Tilburg.EDR. 1993. EDR Electronic Dictionary Technical Guide. Japan Electronic DictionaryResearch Institute.Hajicova, Eva, and R. Alexander. 1994. Machine Readable Dictionary as a Source of134Grammatical Information. In A. Zampolli, N. Calzolari, and M. Palmer, eds.,Linguistica Computationale, Vol. ax Current Issues in ComputationalLinguistics: in Honor of Don Walker, 191-199. Dordrecht: Kluwer AcademicPublishers.Ide, Nancy, and J. Veronis. 1995. Encoding Dictionaries. In: Nancy Ide & JeanVeronis , eds., Text Encoding Initiative, 167-180. Dordrecht: Kluwer AcademicPub.Kang, Beommo. 1996. Using the TEI Scheme in Compiling a Korean Dictionary. InALLC-ACH'96 (6), 162-165. Bergen.Kugler, Marianne, K. Ahmad, and G. Thurmair (eds.). 1995. Translator's Workbench:Tools and Terminology for Translation and Text Processing. Berlin: Springer-Verlag.Sperberg-McQueen, C.M. and L. Burnard (eds.). 1994. Guidelines for Electronic TextEncoding and Interchange (TEI P3). Chicago and Oxford.Sperberg-McQueen, C.M. and N. Ide. 1995. The TEI: History, Goals, and2 Future. InNancy Ide & Jean Veronis , eds., Text Encoding Initiative, 5-16. Dordrecht:Kluwer Academic Pub.Zampolli, A., Calzolari, N., and M. Palmer 1994. Linguistica Computationale, Vol.IXX Current Issues in Computational Linguistics: in Honor of Don Walker.Dordrecht: Kluwer Academic Publishers.135Appendix: Example of a Korean lexical entry and its representation inSDML[1] [mil] (—zi-d-_) -avid om	 /mt- el 4 /	 Ai 131 ().1 Litt . [714r1 	 }4 **4]. 7} 1}E- 61 V c'i 711 c',1	 l-}4-1	°1 4.[71-q01 4+] 71-'02471 141E- 011	 tgA1511-1 71-k1- 01-ti er- 74 01 2_--7/11r-1-.[71-qt.	 11/1' Oa°1	 q11-a" 13 1 -Pr	 .7}oa( 01) 14.	 C) 71-kft1-711 51 u1- .	 V3,1"0: 31°1oI 	 .<entry><male> 7}-a- </mane><hwn> 1<pr>	 </pr><gi><pos> tgAl- </pos></g i><si><nut> 1<expl> V1)40171- ti viti.xl	 </expl><usage><us_ex> 41-at 1<us_ex>	 71-k4:1<us_ex>	 71-k1.-.01A1</usage><sayinp<11118>	 Li-el-m<sag_expl>	 E-01	 711 °i-N-E. 4491 711°---T--m</saying><saying><sag> 7PC1:1 	 t-01.2:-Y;<sng_expl>	tiL c o f 	 41 -a- 4 1 -Pr tre</saying><idiom><ids> 71-k1-( 01) -t--q-<idiuexpl> Cs 49:4711 514. ©z 1 1_- 3.1 	471 °i	 .</idiom><altirs><Mr><altjur_rel> 1(51</al t_hw></si><wfext><hs>	 <Ars><pos> *A1- </pos><hrfext>/miry> 136

A Logical Structure for the Construction of Machine Readable Dictionaries

DSpace at Waseda University

https://waseda.repo.nii.ac.jp/?action=repository_action_common_download&item_id=28531&item_no=1&attribute_id=101&file_no=1

A Logical Structure for the Construction of Machine Readable Dictionaries

Abstract

Similar works

Full text

Available Versions

Waseda University Repository

DSpace at Waseda University