Ramer, Arthur

Czech digital mathematics library

Fuzzy information and combinatorial inequalities

Institute of Mathematics AS CR, v. v. i.

KybernetikaArthur RamerFuzzy information and combinatorial inequalitiesKybernetika, Vol. 28 (1992), No. Suppl, 4--11Persistent URL: http://dml.cz/dmlcz/124198Terms of use:© Institute of Information Theory and Automation AS CR, 1992Institute of Mathematics of the Academy of Sciences of the Czech Republic provides access to digitizeddocuments strictly for personal use. Each copy of any part of this document must contain theseTerms of use.This paper has been digitized, optimized for electronic delivery and stamped withdigital signature within the project DML-CZ: The Czech Digital Mathematics Libraryhttp://project.dml.czSUPPLEMENT TO KYBERNETIKA VOLUME 28 (1992), PAGES 4-11 FUZZY INFORMATION AND COMBINATORIAL INEQUALITIES ARTHUR RAMER Properties of information measures both in fuzzy theory and in the theory of evidence are closely related to combinatorial identities and inequalities. It is demonstrated how study of information func-tions leads to new combinatorial results. Three cases are presented - metric property of fuzzy information distance - approximations of continuous fuzzy information - maximum for a combined Dempster-Shafer evidence measure 1. I N T R O D U C T I O N In 1982 Higashi and Klir [3,6] introduced the notion of fuzzy uncertainty measure and the related fuzzy information distance. They were expressed as function of t he possi-bility assignments corresponding to fuzzy sets under consideration. Later, in [10] and [7] uncer ta inty measures were classified and characterized axiomatically. These char-acterizations were mostly in terms of certain identities, but also included at least one inequality, corresponding to the property of subadditivity. Following tha t development, we [8] obtained related characterizations for information distances. A closer analysis of their propert ies led to a novel combinatorial inequality. In particular, this inequality was responsible for the metric property of G-distance. Another development was related to our recent paper showing how to define notions of fuzzy information on continuous domains of discourse. A natural question thus arose, whether the 'cont inuous ' information can be approximated by the discrete ones. The answer is positive, leading to an interesting family of expressions for limits of sums, representing discrete information values. Very recently, Klir [6] studying general properties of information across various modes of definition - probabilistic, fuzzy and Dempster-Shafer - introduced a principle of 'in-formation preservation' . This study suggested a very intriguing inequality for Dempster-Shafer theory [11]. It gives an upper bound for a given evidence assignment. In the remainder of our paper we outline these combinatorial results. The theorems about rearrangements and about approximations of continuous information will be only i l lustrated with representative examples, the proofs being published separately. The re-sult about 'evidence ' information is very recent and we outline its proof. In the closing we remark that our study is likely only a beginning of mutual influence of the developments in two very distinct disciplines - fuzzy reasoning and combinatorics. Fuzzy Information ami Combinatorial Inequalities 5 2. T E C H N I C A L N O T E In our discussion we often refer to the values of a finite sequence (p,) as arranged in descending order of their values. We denote such rearrangement (p;). On occasion we assume tacitly tha t we have also defined po = p n+, = 0. We adopt [5] style of writing summat ion symbols without explicit limits if the maximum range is mean t . Such range is usually 1 to n, but it may become 1 to n — 1 or 2 to n to avoid terms like logO or 1. Logari thms are in base 2, except in the section on continuous information where natural logari thms are more convenient; we assume that OlogO = 0. Fuzzy sets [1] are defined by their membership grade with respect to a domain of dis-course. We assume these domains to be finitely measurable sets. Most results will be s ta ted only for finite domains; an obvious exception will be the definition and approxi-mat ions of 'cont inuous ' information functions. Given a domain X = {x\,... ,xn} and a membership grade x : X —» [0,1], its values p\ = TT(X\),. .. ,pH = n(xn) form a possibility distribution. They can be interpreted as giving numerical expression of the likelihood of observing a specific instance Xi within the domain X. Given two such distributions ir = (p,) on X and w = (qi) on Y, we can form a joint distr ibution p = jr <g> n? on X x Y by put t ing P- (xi,yj) >-» min(p. ,o ,) • or r „ = min(p.,ijf,). Conversely, given an arbi t rary distribution a = (sij) on a product space X x Y, we can construct its marginal projection a' on X and a" on Y a'(xi) = m a x ^ i j ) <r"(j/,) = max(s,_,). Our model of Dempster-Shafer theory [11] will consist of a domain of discourse given as a finite set A' and of a basic assignment of evidence - an assignment of nonnegative weights nii, 0 < trij < 1 to a family of subsets A; of X. We s t ipulate tha t ]T m, = 1. The subsets A, are called focal subsets of evidence assignment, thus defining a mapping p : Ai H-» m,-. It is convenient to extend p to all subsets of X by pu t t ing p(B) = 0 for a non-focal B. We do not presuppose any containment or intersection s t ruc ture of sets Ai - they may, and usually will have uon-empty intersections; if Ai C Aj it is possible tha t p(Ai) > p(Aj) and so on. 3. INEQUALITIES AND FUZZY INFORMATION We shall conduct our discussion in terms of a finite domain of discourse X and possibility dis t r ibut ions, say it : X —» [0,1], s u p x g X n(x) = 1, represented as a sequence p\,... ,pn, where p; = ir(xi). Here and in the following we assume tha t the cardinality \X\ = n. The basic information function [3, 4, 8] is [/-uncertainty. Given 7r sorted in the descending order of values p~\,... ,p„, we assign it an information value U{ir) = Y,{Pi ~ Pi+i) loS * = J2 P' los yzy • Given two distributions 7r = (p,), p = (r;) such that Pi < rt, i = l , . . , , t ) w e define their information distance g(n,p)=U(p)-U(ir). In a general case of arbitrary 7r and p we first define their lattice-theoretic supremum 7r V p = (g") where o" = max(p;,r;). Now we can define a metric distance G(ir,p) = g(%, 7r V p) + g(p, IT V p). Dual definition, using 7r A p, leads to H(TT, p) = g(it A p, w) + g(w A p, p); this distance is not a metric, however it has an attractive property of being additive. Closer analysis of G(ir,p) shows that its metric property is a direct consequence of an important inequality concerning rearrangements of sequences. Given a sequence (a;) we shall denote its descending rearrangement by a = (a,). Given two sequences a = (a;) and b = (&,) we denote by a V 6 the sequence (maxja,, 6,)) and by a A 6 the sequence (min(a,-, &,)). We observe here that, for example, a V b is a very different sequence from a V 6, as the former results from rearranging the pairwise maxima max(a,, &,) of the original a and b. We can now state the inequalities. We shall state them only in the form directly applicable to the discussion of information metric; their general form is presented in [9]. We assume that a = (a,), b = (&,) and c = (c,) are arbitrary sequences of n elements and that w = (wi) is a non-increasing sequence of weights w\ > • • • > wn. Theorem 1. £ ] w(a) + Y w(b) > Y w(a V b) + VJ ш ( a Л b). Theorem 2. ] P w(a V c) + ] T w(b V c) > ] T w ( a V&) + ]Pu;(č). We now indicate how Theorem 2 leads to the metric property of G distance. Assume given three possibility distributions 7r = (p;), p = (r,) and <r = (a,-). The inequal­ity G(7r,a) < G(7r,p) -f G(p,cr) can be expressed using /7-uncertainties giving, after regrouping terms U(7r V a) + U(p) < U(% V p) + U(p V a). It is effectively the statement of Theorem 2, where to; = log -—j. Fuzzy Information and Combinatorial Inequalities 7 4. CONTINUOUS INFORMATION FUNCTIONS For the continuous case we consider interval [0,1] as the prototypical domain of discourse and function / : [0,1] —• [0,1] to represent a possibility distribution. We recall tha t in the discrete case U(*) = ^2(Pi ~ Pi+i) log« = Yl P{ A log ' and tha t suggests using a 'sorted' version f(x) of the original function f(x), together with the differential d\nx — \dx. (In this section we use In rather than log function.) [5] provide such construction for an arbitrary measurable function / . The rearranged function is defined to ' s tay ' above any given level over the same 'space ' as the original function. It is constructed as a descending function, and its formation generalizes the sorting process of possibility values. We define / ( / ) = / dx. We use it instead of more obvious / dx to jo x J0 x avoid a singularity at 0. Our expression can in fact be viewed as the dis tance to the function which is identically 1 and represents a possibilistic uniform distr ibut ion. In the discrete case it would correspond to the distance to a distribution consisting of \X\ = n values 1. Thus for 7T : X -* [0,1] we put l(ir) = U(\,..., 1) -U(ic) = \ogn-U(w). (To define a general distance, we put for fy < / 2 9{fnh)~ I Jo lUx)~Һ(x)dx and then extend it to arbitrary fi,fi [3].) If the domain of discourse is an arbi t rary set X t h e rearrangement results in a descending function over an interval corresponding to t h e measure of X. It is then only required to change limits of integration to correspond to t h e end-points of t h a t interval. Our preliminary results indicate t h a t the complete theory, built for discrete distribu-tions, carries to the sett ing of continuous (or measurable) distr ibutions. It is significant t ha t all the information values can be obtained through approximations by discrete dis-t r ibut ions. It means tha t the discrete cases can be viewed as ' imperfect ' approximat ions of an idealized continuous description. E x a m p l e 1. Let us consider possibility distributions represented by f(x) = xk, k = 0 , 1 , Denoting Jk = I(xk) and remembering tha t xk = (1 — x)k, we find l~U = o = I(\)= f Jo /" - - (1 -jo -j2 - / ( x3) = f L l í i Z f ) ! ^ = f(2- X)dx -1 Jo x Jo z J j = I(x) ~ I Ü ^-dx - 1 To find a general expression for J/t, we first compute Jk — J/t-i / " ( i - ( i - » ) ' ) - ( i - ( i - * ) * - ' ) ^ = fi'V-c-^.Ai.,^ ' jo a: 7o * As J 0 = 0 we have J* = 1 + 1 -J 1- | = H*, the Arth harmonic number . Finally, we remark tha t / ( / ) can be approximated as a limit of / (7r ( n ' ) , where 7r(n> are discrete distr ibutions approximating / . Already a non-trivial example is offered by a linear function. E x a m p l e 2. We select f(x) = 1 — x and approximate it using the values at j j , .~ , . . . 0 . T h e approximat ing distributions are 7r(n' = ("—^, s ~-^ , . . . 1) and have as their information measures t!(T(n)) = 5>- P t + 1 ) ln* = = E(^-^>- ;• = — > In J = — In n! n -—' n Invoking Stirling's formula, we find In n! = n In n — n + 0 ( l n n) , and therefore U(7r(n') ~ Inn — 1, n —> oo. This gives /(7r ( n ' ) = l n n - ( / ( 7 r ( n ) ) ~ l which agrees with / ( / ) . A more general f(x) = x* leads to an interesting identity involving limits of logarith-mic sums. Approximat ing as above we get £/(*<»>) = £ ( # - & . > . - . . z-< n* Expanding with respect to n — i and retaining only the highest order terms gives Subs t i tu t ing into l i m ^ o o Z(7r(n') = I(xk) finally leads to 11-.00 y n J Fuzzy Information and Combinatorial Inequalities 9 5. INFORMATION INEQUALITIES IN DEMPSTER-SCHAFER THEORY We consider a finite domain of discourse X, with cardinality \X\ = n, endowed with a basic evidence assignment fi concentrated on the focal subsets Ai C X. We put fi(Ai) = m„ where 0 < m,- < 1 and )> m,- = 1. For this theory there were developed essentially two separate information measures [1]. The first one - nonspecificity N(fi) expresses how specific is the assignment. It is represented by a sum At(/.) = £m,log|/l,|. The other one - S(ii) addresses the question of how much overall information results from mutual support lend by the focal subsets. Given a focal set B we first define its plausibility Pl(B) = ^ m«- N o w S^) = ~ H m ; loS Pl(Ai)-AinB^t Klir [6] has proposed studying a combined information measure \M JM = од + sм = $>..og^щ-He in particular suggested that this measure has as its maximum value log \X\ (for a given overall domain X), and asked for a description of evidence assignments which have this information value. The answer is given in the next two theorems. We let F be a concave, monotonically increasing function which satisfies limj:_o xF(x) — 0 and is defined for all positive real numbers. Thus if F(x) is twice differentiable, then F'(x) > 0, F"(x) < 0. An example of such function is logx, and conclusions apply to the question as posed. Theorem 3 . P r o o f . Function F being upward concave and m,'s summing to 1, we can apply Jensen's inequality [5] and obtain the first inequality. We note that the equality occurs u . i here only if all terms ' are equal. '. '' As F is monotonically increasing, the other inequality will follow if we show that We rewrite the left-hand side as . ™iAi) S>i PI(AІ) and consider it a counting expression for a multiset, where elements are those from all >t;'s separately, and where every x € X is entered as many times as the number of 10 A. RAMER sets At to which it belongs. However, we shall count each element only with a weighted multiplicity - if it is contributed by A, we give it weight ' . . Pl(Ai) Now the combined multiplicity of an element x : £ X is E m, PUAi) Let the summat ion be over the sets A\,..., A),. As x is simultaneously in all A\,..., A^, they all have pairwise non-empty intersections and all contribute to one another plausi-bilities. In part icular Pl(Ai) > m(A\) + • • • + m(Ak) and the combined multiplicity of any given element a: € AT is k k y^^<Y ^ = 1 ~{ 'W ~.{ztmi + ' ' ' + m* They are n elements in X and thus the sum of their multiplicities is less or equal to n - the cardinality of A'. We remark tha t the equality here requires tha t (in the notat ion above) Pl(Ai) = m\ + • • • + m * for all A,. This would imply tha t any set from among all Aj t ha t intersects A; must contain the selected element x. As x could be any element of Ai it means tha t A,- would be a subset of every Aj it intersects. This is possible only if AiS are all disjoint. Finally, all x £ X must be accounted for; therefore Ai's form a par t i t ion of X. • '• T h e o r e m 4 . Combined information value of the evidence assignment J(n) reaches its max imu m value log \X\ exactly when it consists of disjoint focal subsets Ai covering X and assigns weights m, proportional to cardinalities |A.,|. Thus A,r\A3 = %, ifj, \jAl = X, m, = fi(Ai)=1-^ P r o o f . Immedia te from the previous discussion. • A C K N O W L E D G E M E N T The author wishes to thank the anonymous reviewer for a meticulous study of the paper. REFERENCES  [1] D. Dubois and H. Prade: Possibility Theory. Plenum Press, New York 1988. [2] D. Dubois and H. Prade: Properties of measures of information in evidence and possibility theories. Fuzzy Sets Systems 21, (1987), 161 - 182. [3] M. Higashi and G. Klir: On the notion of distance representing information closeness. Internat. J. General Systems 9 (1983), 103 - 115. [4] M. Higashi and G.Klir: Measures of uncertainty and information based on possibility distributions. Internat. J. General Systems 8 (1982), 43 - 58. Fuzzy Information and Combinatorial Inequalities 11 [5] G. Hardy, J. Littlewood and G. Polya: Inequalities. Cambridge University Press, Cambridge 1934. [6] G. Klir: A principle of uncertainty and information invariance. Internat. J. General Systems 11 (1990), 249 - 276 . [7] G. Klir and M. Mariano: On the uniqueness of possibilistic measure of uncertainty and information. Fuzzy Sets Systems 2 (1987), 197 - 220. [8] A. Ramer: Structure of possibilistic information metrics and distances. Internat. J. General Systems 17(1990), 21 - 32, and 18 (1990), 1 - 10. [9] A. Ramer: Certain Inequalities Related to Rearrangements. Technical Report OU-PPI-TR-89-01, University of Oklahoma, Norman, OK 1989 (submitted to Eur. J. Combinatorics, 1991.) [10] A. Ramer and L. Lander: Classification of possibilistic uncertainty and information functions. Fuzzy Sets Systems 24 (1987), 221 - 230. [11] G. Shafer: A Mathematical Theory of Evidence. Princeton University Press, Princeton 1976. Prof. Dr. Arthur Ramer, School of Electrical Engineering and Computer Science, University of Ok-lahoma, 200 Felgar St., Room 111,, Norman, OK 73019. U.S.A. 

English

http://dml.cz/bitstream/handle/10338.dmlcz/124198/Kybernetika_28-1992-7_1.pdf

Fuzzy information and combinatorial inequalities

Abstract

Similar works

Full text

Available Versions

Czech digital mathematics library

Institute of Mathematics AS CR, v. v. i.