Jeon Mee-Sun

Park Se-Young

English

Jeon, Mee-Sun

49172

Park, Se-Young

49173

Waseda University Repository

Document Ranking Method for High Precision RateMee-Sun Jeon and Se-Young ParkNatural Language Processing SectionElectronics and Telecommunications Research InstituteTaeJon, Korea{msjun,sypark}@com.etri.re.krAbstract Many information retrieval(IR) systems retrieve relevant documentsbased on exact matching of keywords between a query and documents. This methoddegrades precision rate. In order to solve the problem, we collected semanticallyrelated words and assigned semantic relationships used in general thesaurus and aspecial relationship called keyfact term(FT) manually. In addition to the semanticknowledge, we automatically constructed statistic knowledge based on the concept ofmutual information. Keyfact is an extended concept of keyword represented by nounand compound noun. Keyfact can be a verb and an adjective including subject orobject term. We first retrieved relevant documents with original query using tf * idfweighting formula and then an expanded query including keyfacts is used in bothsecond document ranking and word sense disambiguating. So we made animprovement in precision rate using keyfact network.1 IntroductionThe general users are more interested in concepts rather than words itself. But manycommercial IR systems retrieve relevant documents based on keyword string matching betweena query and documents. There are two problems in using the method. The first problem is thatwords are ambiguous, and this ambiguity is causative of retrieving irrelevant documentsemantically. Therefore lexical ambiguity has to be resolved. The second problem is that adocument is treated as a irrelevant document in spite of a relevant document, for the documentdoes not include the same words as query terms. So an original query has to be expanded tosemantically related words.In order to solve the problems, we consider keyfacts as well as keywords. A keyfact is anextended concept of a keyword and can be defined as a verb and an adjective which are everyprobability that occurs in several times based on threshold value. We collect semanticallyrelated keyfacts from an encyclopedia and assigned semantic relationships using in generalthesaurus and a special relationship of FT. We use the semantic informations for documentranking and word sense disambiguation. The remainder of this paper is organized as follows.Section 2 gives a definition of the KN and the KN construction method. Section 3 describesword sense disambiguation. The results of performance comparison are presented in section 4.The concluding remarks are described in section 5.2 Keyfact Network2.1 The Definition of KNIn this paper, we collected words and their semantically related words from anencyclopedia(ENCY). Collected informations are used to understand user's request. The KNconsists of nodes and edges. Nodes are defined as words rather than word sense and the edgesrepresent binary relationships, such as BT(Broader Term), NT(Narrow Term), RT(RelatedTerm), HP(Has Part), UF(Used For), and FT(keyFact Term). FT is defined as relationship255between noun and verb or adjective. Most IR systems have used to the relationships except FTrelationship in their thesaurus. FT is effectively used to resolve lexical ambiguity and computequery-document similarity. KN has a semantic and statistic informations like this format'relationship(freq(x), freq(y), freq(x,y))'.2.2. Construction of keyfact network(ENCY) explains matters systematically. The ENCY has two characteristics. First, it hassyntactic characteristic composed of title word and it's explanation part. Second, it has semanticcharacteristic that most words in the explanation part are semantically related with a title word.ENCY is good to easily collect words and it's semantically related words. We thought thatENCY is a proper text for construction of semantic informations. First, we manually markedkeywords like nouns and compound nouns which are semantically related with a title wordwithin each explanation, and we marked keyfacts like verbs and adjectives which are includeadjacent subject or object if possible. Second, we assigned semantic relationships using ingeneral thesaurus such as BT, NT, RT, HP, UF and a special relationship of FT. Third, wecompute each frequency and co-occur frequency between keyfacts in ENCY. There are 17%ambiguous words of about 22,000 title words. 88% of the ambiguous words are two sensewords. KN has 88,010 whole entries, and each entry has an average of 7.3 related words.For example in the query "V .7} 11 7+Tit? (What's the number of letter in theTripitaka Koreana) ", the word '44' might be one of three meanings in Korean language. Thethree meanings are the number of letter, self-surrender, and embroidery in Korean language.But KN does not have the semantic 'the number of letter°, because input text of KN is smallcorpora. Therefore the KN must be changed and expanded on demand a new ENCY. Whenevera new document like an encyclopedia is occurred, we don't have to process first and second stepfully manually. Keyfact extractor extracts noun, compound noun, and original form of verb. Ifthe words like homonym and polysemous word exist in a new document, disambiguator willreduce humane intervention.3 IR System Based on Keyfact NetworkFigure 1 depicts the structure of our IR system. The IR system based on the KN have anadvantage. The edges of the KN represent term dependencies more exactly than theconventional statistical measures. So retrieval effectiveness of IR systems can be improved byusing KN. Input of our system is natural language query and output is ranked documents.In our system, we have 5 modules. The system has keyfact extractor, query expander,disambiguator, document ranking subsystem and IR engine. In most other systems nouns(N)and compound nouns(CN) are considered as prime keywords. In this paper, we extract verbs(V)and adjectives(AD) as well as N and CN from queries. Our keyfact extractor extracts N, CN,original form of V and AD. Disambiguator resolves lexical ambiguity in a query. Queryexpander expands keyfacts extracted from original query into semantically related keyfacts.Expanded queries are consisted with BT, NT, HP, UF, RT, and FT relationship. In IRsubsystem we execute retrieve with original query. We use expanded queries in documentranking and applying tf * idf weighting method. Because query expansion is recall enhancingtechnique, we used expanded query keyfacts when compute query-document similarity. So wegot high precision rate.3.1 DisambiguatorBefore query expansion, if a query has polysemy or homonym, their lexical ambiguity have tobe resolved. In our disambiguator, we used a knowledge base KN.The literature generally divides lexical ambiguity into two types: syntactic andsemantic[2]. Syntactic ambiguity refers to differences in syntactic category. Semanticambiguity refers to differences in meanings. A number of approaches have been taken to word256AmbiguatedOriginal QuerynIFirst Retrieved Documents'Expanded Queries sense disambiguation. Lesk uses the Oxford Advanced Learners Dictionary[3] and Weiss usesword cooccurrences[4].In this paper, we resolved semantic ambiguity of N using KN. The ambiguity in a query isresolved when the query is analyzed. Sense resolution is approximated by requiring new termsto be related to at least two original query terms. Shorter queries benefit less than longerqueries. Ambiguous words are not able to effectively expand before the ambiguity is notresolved. We resolve ambiguous query terms, and then expand unambiguous query terms. Weprepare tables. The number of tables is same as the number of meanings of an ambiguous wordregistered in KN. If each keyfact lists of each meaning has common keyfact with keyfact listsof the other keywords in a query, the value is simply incremented. A meaning having maximumvalue over threshold value is chosen as a proper meaning. Let's take the following examplesentences.Example 1) r-oh--fi	 rasii	 114-.,Q°.171-?(What's the difference donkey and horse?)Example 2) 4%)-01 131- 21' °It	 t--Frt?(What kind of tea do man can drink ?)In the example 1, keyfacts extracted from the query are donkey, horse, and difference. Koreanlanguage ig is a polysemous word and has several meanings, which are a language, a horse, theend, a unit of measure, etc. la's semantic is to be animal according to common keyfact lists inKN.Natural Language QueryISecond Retrieved Documents<Figure 1> The structure of our IR systemBecause tg: has four common keyfacts donkey, Equida, tail, and animal, the meaning of isa horse. In this manner keyfacts extracted from the example 2 are man, drink, tea, and a kind.meaning is to a tea according to keyfact drink. In the example 1, keyword concept wasuseful to disambiguate. In the example 2, keyfact concept was useful.3.2 Query ExpansionGiven a KN, there are a wide choice of words to add to a query vector. One can add only thesynonyms, or synonyms plus all descendants, or synonym plus parents and all descendants, or257synonyms plus directly related words, etc. and any number of child links may be traverse.Expansion by synonyms plus and directly related word is benefit[1]. So we choose theparameter. As an example of the expansion processing, let's consider the related words for'quartz'. If the child's links are limited to depth one, then mineral, rock, be given a pressure,compose rock, like a glass, crystal, and be changed white sands would be added.3.3 Document RankingIR systems must be designed to aid users in determining which documents of those retrievedare most likely to be relevant to given queries. Therefore document ranking is very importantpart. Most commercial text retrieval systems employ inverted files to improve retrieval speed.The inverted file specifies a document identification number for each document in which theword occurs. In order to improve retrieval effectiveness, vector processing systems employingsimilarity measures have been suggested and studied extensively. In a vector processingsystem, An expanded query(EQ) can be represented as vector <q 1 , q2, qv> with originalquery terms and one depth descendants in KN. The similarity between EQ and documents canbe computed in order to rank the retrieved documents in decreasing order of the query-document similarity [5,6]. Dij represents the weights of term j in document i. Qj represents theweights of term j in query q. tfDi(tj) represents the term frequency of term tj in document Di.idf(tj) is called the inverse document frequency of term tj and is set to log2(N/df(tj )). N is thenumber of documents in a collection df(tj ) is the document frequency of term tj.Similarity( EQ, D i) = E D i; Q.;	 (1)j=1Dij = tfDi(tj) * idf(tj)= tfDi(tj ) * log2(N/dfj)	 (2)Each document vector uses tf * idf weighting strategies. Wii in equation 1 is computed byusing equation 2 and saved in a inverted file. Thus a term has a high weight in a document if itoccurs frequently in the document but infrequently in the rest of collection. The vectorprocessing system allows a query to be expressed as a natural language text describing theuser's information need. The description can be treated as a short document so that Wqj can beexpressed in ft * idf weights as well. But to further reduce the computational cost, the weight oforiginal query terms is 0 when a term is absent, and 1 when a term is[?]. The weight ofsynonym of original query terms is 0.8, and The weight of all descendants of original queryterms is 0.3.In our document ranking subsystem, First we compute original query-document similarityusing equation 1. Because is already computed in the inverted file, we retrieve relevantdocuments by computing only Q. Until now we have considered keywords, not consideredkeyfacts and lexical ambiguity of keywords in a query and documents. Generally expandedqueries degrade the precision rate and query expansion is a recall enhancing technique. So wecontrolled order of already retrieved documents using EQ, which includes keyfacts as well askeywords. For each keyfact in the expanded queries, the system enters the document in a hashtable; the table is keyed on the document number, and the value is initially 1. If the documentswas previously entered in the table, the value is simply incremented. The end result is that eachentry in the table contains total number of keyfacts of expanded queries that occurred in thatdocument. The table is then sorted to produce a ranked list of documents.4 Performance Comparison258We collected about 600 natural language queries from general users and choose 100 well-formed queries. The rest queries don't have relevant documents in our collection and ill-formedqueries. We evaluate the performance of the precision based on KN. The test collection is anencyclopedia and has about 22,000 documents. When we don't use the KN, the precision rate is66%. when use, the presion rate is 88.8%. Following examples show retrieved documentsprofited by considering expanded query keyfacts in query-document ranking. Italic style fontsrepresent relevant documents. In a query 'Einstein's biography T, When EQ was considered insecond document ranking, irrelevant relevant documents ' Einstein' was ranked first order.5 ConclusionMost IR systems do not consider lexical ambiguity of query terms, document terms and alsoindex terms. And consider nouns and compound nouns kwyword by indexing word. Verbsand adjectives are useful indexing words. In this paper, we construct the Keyfact Networkwhich is a kind of semantic network. Keyfact Network provides binary relationships such asBT, NT, RT, HP, UF, and FT. FT is a relationship noun-verb pair, noun-adjective pair, verb,adjective. At this time verb and adjective is restored to the original form. We resolved lexicalambiguity of query terms and improved the precision rate by considering EQ in the seconddocument ranking.In the future, we will expand the proposed Keyfact Network with a new ENCY, besidesthe encyclopedia have to experiment another test collection, and use the Keyfact Network inautomatic indexing.References[1] Ellen M.Voorhees, "Query Expansion using Lexical-Semantic Relations", Proceedings ofthe Association for Computing Machinery-Special Interest Group on Information Retrieval,Dublin, pp.61-69, 1994.[2] Robert Krovertz and W.Bruce Croft, "Lexical Ambiguity and Information Retrieval",Association for Computing Machinery Transaction on Information Systems, Vol. 10, No. 2,pp.115-141, 1992.[3] Lesk M, "Automatic sense disambiguation using machine readable dictionaries: How to tella pine cone from an ice cream cone", Proceedings of Special Interest Group for Documentation,pp.24-26, 1986.[4] Young S. Han, Young Kyoon Han, and Key-Sun Choi, "Lexical Concept Acquisition FromCollocation Map", Proceedings of a Workshop Sponsored Group on the Lexicon of theAssociation for Computational Linguistics, Ohio, pp.22-31, 1993.[5] Gerard Salton, Automatic Text Processing, Addison-Wesley Publishing Company, NewYork, pp.229-271, 1989.[6] Roy Rada and Judith Barlow, "Document Ranking Using an Enriched Thesaurus", Journalof Documentation, Vol. 47, no. 3, pp.240-253, 1989.[7] Yih-Chen Wang and James Vandendorpe, Martha Evens, "Relational Thesauri inInformation Retrieval", Journal of the American Society for Information Science, Vol. 36, No.1, pp.15-27, 1989.259260

Document Ranking Method for High Precision Rate

DSpace at Waseda University

https://waseda.repo.nii.ac.jp/?action=repository_action_common_download&item_id=28512&item_no=1&attribute_id=101&file_no=1

Document Ranking Method for High Precision Rate

Abstract

Similar works

Full text

Available Versions

Waseda University Repository

DSpace at Waseda University