Iványi, Antal M.

Sotnikov, A. N.

English

Iványi Antal M.

Sotnikov A. N.

University of Szeged

On the optimization of library information retrieval systems B y A . M . IVANYI a n d A . N . SOTNIKOW Introduction In some information retrieval systems (e.g. in many library ones) the content of documents (books, papers, patents, computer programs etc.) and the content of user's questions are characterized by using a given set of key-words (so called des-criptors). The aim of the retrieval is to find all the documents whose content is charac-terized by the descriptors in the question and, may be, by other descriptors too. Of course, the sequential retrieval represents a slow method, which is satis-factory only in small systems. A more effective method is proposed by G. Salton [1]. According to this method the set of documents is decomposed into disjoint subsets (so called clusters) consis-ting of "similar" documents, and the retrieval is organized in a hierarchical manner. Recently an excellent review was publicated by S. T. March on the different retrieval methods [2]. According to Salton's proposition [3] the efficiency of the retrieval system is characterized by the expected waiting time of the users, therefore the aim of the optimization is to minimize this time choosing suitable parameters (number of clusters, sizes of clusters, distribution of documents among the clusters). At first in § 1 we describe a mathematical model of such systems [1], then in § 2 the mentioned expected time is computed for some values of parameters, later in § 3 this time is analysed as a function of the different parameters, and finally in § 4 some practical conclusions are made. § 1. The mathematical model Let m, n, r and z be positive integers, <5 = {AD ..., JD;, •••, A.-i} — documents, ...,Kj, ...,Km— key-words, <?={Ôo> ..., Qp,..., Qr-J — possible questions of users, a={A0,..., Ap,..., — the corresponding answers of the information retrieval system, where Qp^x, Ap\i<5 for />=0, ..., r— 1. The content of documents is characterized by the document description matrix D=[<y„xm> where d u = l , if Kj characterizes Du and d u = 0 otherwise. The con-tent of questions is characterized by using the question description matrix Q=[qPj},xm, 6 * 324 A. M. Ivanyi and A. N. Sotnikow where qp~ 1, if Kj characterizes Q,„ and qpJ=0 otherwise. The /-th row d( of D represents the description of Dh and the /;-th row qp of Q represents the description of Qp. In the case of the question Q„ we have to find the documents Dh for which The set 5 is decomposed into disjoint clusters C0, ..., C\, ..., The de-composition is defined using the decomposition matrix Y=[y t k]H X Z , where ^ = 1 , if D£Ck and yik-0 otherwise. The content of clusters is described by using the characteristic matrix V=[ukJ]2>tm, where ukj=0 if for any D£Ck, dtJ~0, and w t .= l otherwise. The k-th row ufc of U is called the characteristic vector of Ck. 1 The retrieval consists of two levels: on the first level the description qp of the question Qp is compared with the characteristic vectors ufc for k=0, ..., z - 1 . On the second level a sequential retrieval is realized in the relevant clusters: a cluster C„ is relevant to Qp iff u4£qp (uktj^qpj for j=0,1, ..., m-1 ) . As a time unit (and as a cost unit, too) let us choose the time required to com-pare two m-dimensional vectors. Then in the case of a decomposition matrix Y the met SI)Y of the answer Ap to a question Qp is defined by SPY = z+'2\Ck\Ltf+t'2LkP, k=0 k=0 where Lkp- 1, if Ck is relevant to Qp and Lkp=0 otherwise, t is a nonnegative real parameter (the time, required to prepare a new cluster to the processing). The effi-ciency of a given decomposition algorithm A, resulting a decomposition matrix Y^ is denoted by M(Y4) and defined by the average cost of answers to all possible questions ' p = 0 For example, let m=2, n=r-4, z=2, t=0, '0 0 ' 'O O' D = 0 1 1 0 and Q = 0 1 1 0 ,1 1. 1, Then the corresponding answers are A0={D0, Du Da, D3}, A1—{D1, A2={D%, D3} and A = {Dai-Let us assume that we construct two different decompositions defined by the matrices '1 0] '1 cf 1 0 and Y2 = 0 1 1 0 1 0 .0 1, ,1 0, Then the corresponding characteristic matrices Ux and U2 are as follows; * - ( » } ) and * _ ( } J). On the optimization of library information retrieval systems 325 If all the four possible questions are put once, then we have eight comparisons on the first level for both decompositions; the second decomposition requires 4 x 4 = = 16 ones on the second level, while the first needs only 2 x 4 + 2 x 2 = 1 2 , there-20 24 fore, M(YX)= — =5, while M(Y2)= — = 6. § 2. Efficiency of a decomposition algorithm Let m be fixed, n=r—2m. Let us suppose, that the descriptions of the docu-ments (the rows of the matrix D) and the descriptions of the questions (the rows of the matrix Q) are different, that is 1 implies d ^ d , , and =r— 1 implies qpT^qp. In this case we have 2m possible different documents and also 2m different questions. For the simplicity let us suppose, that the descriptions as binary numbers give the corresponding indices, that is m — 1 m — 1 i = 2 d t j l " - 1 - ' , p = z qPJ2m-1-j 0, p = 0, . . . , 2 m - l ) . J=0 j=0 Let us suppose that x is a nonnegative integer and |CJ=2'x for k~0, 1, ... ...,2—1. In this case we have z=2m~x and O^xSm. For this special values of parameters the decomposition algorithm S [4] is defined as follows: ck = { A t - 2 * , . . . , X ) f c . 2 x + 2 * _ i } ( k = 0 , 1, . . . , 2m~x—l). Now we can give the efficiency of S as a function of parameters x, m and t [4]. Theorem 1. If x,m are positive fixed integers, n=r=2m, t is a nonnegative real number, implies d ^ d j , 0 ^ / ; i m p l i e s q p ^q P , then M(YS) = 2m-x+22x-m2,m-x + t-2x- 'n3m-x . • (1) Proof. According to the definition M(YS) we have M(YS) = ^ [22m~x + 2x 2 2 1 + ' 2 1]. ^ * = 0 qpsak k = 0 q p s u k In this expression the sum ^ 1 is equal to the number of questions for which (qP;o, ..., = Considering ufc as a binary number bk according to the grouping rule of S we get m - 1 bt= 2 ukJ2'»-j-i = (k+l)2x-l o (the first m—x digits determine k, the last x digits are l's). If u t contains w l's on the first m—x places, then we have 2W-2X questions with Therefore, using the equality 326 A. M. Ivanyi and A. N. Sotnikow we get m—x (m — M(Ys)=2m-x+(2x~m + t-2-m) 2 w \T"2X= w=0 V W t § 3. Analysis of the influence of the continuous parameters According to the Theorem 1 the cost M(YS) depends on x, m and t as M = M(x, t,m) = e">>'e-xb + em^-b)ex(2b-c) + (tem(c-b)e^x(c-b)^ where b=In 2% 0,69314718 and c=ln 3 ^ 1,09861229, t is a nonnegative real parameter, m is a positive integer parameter, and x represents the independent variable being integer and 0 S x 5 f f l . In the practice the number of key-words m equals 10—1000 [4, 5], the time of preparation of a new cluster to the processing t is about 0—109 time units (may be, it is necessary to put new disc or magnetic tape). In the analysis of the function M = M (x, i, m) its continuous variant R = =R(x, t, in), plays an important role, where x, t and m are real variables with x € ( - ~ , ( • -» , +oo), m£(-co, +°o). Lemma 1. If and m are fixed real numbers, then the function R = R(x, t, m) as a function of x is convex, it has one local minimum and this minimum represents an absolute one too. • Proof. Differentiating R(x, t, m) by x we get R x =-b.embe-bx+(2b-c)em(c-b)e*(2b-c)-(c-b)t-em(c-b'>e-xic-''\ Since .Rx is a monotone increasing continuous function of x (its members are increasing) and its limit is + ° ° f o r x—+ <=o and — °° for x— — °°, so R* has a unique zero-place x0 and R j < 0 for x < x 0 , R ^ O for x > x 0 . Therefore R(x, t, m) is convex, and it has an absolute minimum at x0 . • Lemma 2. For a fixed m the function x 0=x 0 ( f , m) is a monotone increasing function of t. • Proof. From R t = 0 we get For a fixed m the function /(x0, m) is a iiionotone increasing one of x0 for x0€( — s o it has a monotone increasing inverse x0=x0(t, m). • Lemma 3. For fixed the function x 0=x 0 ( i , m) is a monotone increasing f u n c t i o n Of /77. • . 2b-c t = t(x0, m) = ——r-c — b b em(_2J>-c)e-x0(2b~c) c-b On the optimization of library information retrieval systems 327 Proof. It follows from R*=0 that m = ^ J — In ^ . ( ¡a-oj 2b-According to Lemma 1 for given t and m there exists a corresponding x0 , there-fore the expression in the square brackets has to be positive (otherwise m has no meaning). Now we have (3fc_c)e(»-«*. Kc-b)(2b-c) eXQ(ib_c) dm b b dXo (2b -c) «».(»-o] The expression in the numerator is greater than the one in the square brackets of the denominator (the positive member is multiplied by 3b—c, the negative one by 2b—c), therefore > 0 and so the derivative - also is everywhere positive and x0 is an increasing function of m. • Theorem 2. If and in are fixed real numbers, then f = 0 m + y , if t = 0, Xo\>fim+y, if / > 0 , In 2 ln -where p= «0,29330495 and y = / " f / 3 « 0,89657440. • In 8/3 In 8/3 Proof. Lemma 1 and the equation R t = 0 at the condition t = 0 imply x 0 = = jSm+y, and then Lemma 2 gives the remaining part of the theorem. • § 4. Conclusions In the previous paragraph we considered x, t and m as continuous parameters. In the practice x and in have positive integer values. According to Lemma 1, R(x, t, m) as a function of x is a convex function, therefore Theorem 2 implies for the optimal size-parameter xopt the assertion: if t - 0, then lpm+y) *opt ^ \pm+y). If 0 then we have a lower bound xop t^[Pm+y\ due to Lemma 2. Analysing the function M(x, t, m) one can see that for small values of t the third member of (1) has a relatively weak influence on the value of xop„ and so in the first approximation is neglectable. We propose the following approximate formula: xopt « 10,29330495m+0,89657440]. (2) 328 A. M. Iványi and A. N . Sotnikow Table 1. Summary of the numerical results m t •̂ OPT 7П t •̂ OPT Xa 20 1W> 7 7 80 10° 24 24 10» 8 7 10s 24 24 104 14 7 104 24 24 10е 20 7 10« 24 24 40 10" 13 13 100 10° 30 30 102 13 13 102 30 30 104 14 13 104 . 30 30 10е 20 13 10® 30 30 60 10° 19 19 102 19 19 10* 19 19 106 21 19 The following table shows some numerical results of a BASIC-program, run-ning on a personal computer Commodore-64. In this table xopt represents the optimal integer value of „v and xa is the value resulted by the approximate formula (2). According to the presented numerical data formula (2) results good approxi-mations of the optimal size parameter for m>20 and 1000000. The authors are indebted to Dr. Péter Simon at Eötvös Loránd University of Budapest for the consultation and valuable remarks. References [1] SALTON, G., Automatic information organization and retrieval. McGraw Hill, New York, 1968. [2] MARCH, S. Т., Techniques for structuring databese records. Computing Surveys, 15 (1) (1983), 45—79. [3] SALTON, G., Dynamic information and library processing. Prentice Hall Inc., Englewood Cliffs, 1975. [4] И в а н и A. M. и С о т н и к о в A. H. Об оптимизации дескрипторных А И П С с зонно-иер-архической организацией поискового множества. Вестник Московского Университета, Сер. 15. Вычислительная Математика и Кибернетика, (2) (1984) 53—57. [5] IVANYI, А. М. and NYIRADI, L., Library retrieval systems I N F and F A R M D O C . In: Proc. of Conf. for information-servicing of professionally linked computer users (Varna, 1977), 307—313. (Received March 13, 1985) 

On the optimization of library information retrieval systems

http://acta.bibl.u-szeged.hu/12400/1/cybernetica_007_fasc_003_323-328.pdf

On the optimization of library information retrieval systems

Abstract

Similar works

Full text

Available Versions

University of Szeged