De Cooman, Gert

Miranda, Enrique

Quaeghebeur, Erik

English

de Cooman, Gert

Ghent University Academic Bibliography

Representing and assessing exchangeable lower previsionsDe Cooman, GertQuaeghebeur, ErikGhent University, SYSTeMS Research GroupTechnologiepark–Zwijnaarde 9149052 Zwijnaarde, BelgiumE-mail: {gert.decooman,erik.quaeghebeur}@ugent.beMiranda, EnriqueRey Juan Carlos University, Dept. of Statistics and O.R.C-Tulipán, s/n28933 Móstoles, SpainE-mail: enrique.miranda@urjc.es1 IntroductionThis paper deals with belief models for both (finite) collections and (infinite) sequences of exchangeable randomvariables taking a finite number of values. When such collections or sequences are assumed to be exchangeable,this more or less means that their specific order is irrelevant. One of the reasons why exchangeability is deemedimportant, especially by Bayesians, is that, by virtue of de Finetti’s Representation Theorem, an exchangeablemodel can be seen as a convex mixture of multinomial models. This has given some ground (de Finetti, 1937,1974-1975; Dawid, 1985) to the claim that aleatory probabilities and IID processes can be eliminated fromstatistics, and that we can restrict ourselves to exchangeable sequences instead.De Finetti presented his study of exchangeability in terms of the behavioural notion of previsions, or fairprices. One assumption underlying his approach is that a subject should be able to specify a fair price P( f )for any risky transaction, or gamble, f . He should therefore always be willing and able to decide, for any realnumber r, between selling the gamble f for r, or buying it for that price. This may not always be realistic,and it has been suggested that we should explicitly allow for a subject’s indecision, by distinguishing betweenhis lower prevision P( f ), which is the supremum price for which he is willing to buy the gamble f , and hisupper prevision P( f ), which is the infimum price for which he is willing to sell f . The resulting theory ofcoherent lower previsions, brilliantly defended by Walley (1991), generalises de Finetti’s behavioural account ofsubjective, epistemic probability, and tries to make it more realistic by allowing for a subject’s indecision.But, even if we allow for a subject’s indecision by considering lower, rather than precise, previsions, itremains very useful to consider what are the consequences of a subject’s exchangeability assessment. This isour motivation for studying exchangeable lower previsions here. The exchangeability assessment has a clearimpact on the structure of so-called exchangeable coherent lower previsions. We show how they can be writtenas a combination of (i) a coherent prevision expressing that permutations of realisations of such collections orsequences are considered equally likely, and (ii) a coherent lower prevision for the ‘frequency’ of occurrence ofthe different values the random variables can take. This is the essence of representation in de Finetti’s sense: wegeneralise his results to coherent lower previsions. We also solve a more practical problem: how to extend anumber of lower prevision assessments to an exchangeable lower prevision that is as conservative as possible.2 Exchangeable random variablesConsider N ≥ 1 random variables X1, . . . , XN taking values in a non-empty and finite setX . A subject’s beliefsabout the values that these random variables X= (X1, . . . ,XN) assume jointly inX N is given by their (joint)distribution, which is a coherent lower prevision PNX defined on the set L (XN) of all gambles (boundedrandom variables) on X N . Let us denote by PN the set of all permutations of {1, . . . ,N}. With any suchpermutation pi we can associate, by the procedure of lifting, a permutation ofX N , also denoted by pi , that mapsany x= (x1, . . . ,xN) inX N to pix := (xpi(1), . . . ,xpi(N)). Similarly, with any gamble f onX N , we can considerthe permuted gamble pi f := f ◦pi , or in other words, (pi f )(x) = f (pix) for all x ∈X N .Now a subject judges the random variables X1, . . . , XN to be exchangeablewhen he is disposed to exchangeany gamble f for the permuted gamble pi f , meaning that PNX (pi f − f ) ≥ 0, for any permutation pi (Walley,1991). Taking into account the properties of coherence (a rationality criterion, see Walley (1991)), this meansthat PNX (pi f − f ) = PNX ( f −pi f ) = 0 for all gambles f onX N and all permutations pi inPN . In this case, wealso call the joint lower prevision PNX exchangeable. A subject will make an assumption of exchangeabilitywhen there is evidence that the processes generating the values of the random variables are (physically) similar(Walley, 1991, Section 9.5.2), and consequently the order in which the variables are observed is not important.When PNX is in particular a coherent (precise) prevision PNX , exchangeability is equivalent to havingPNX (pi f ) = PNX ( f ) for all gambles f and all permutations pi , and this is also equivalent to having pNX (x) =pNX (pix) for all x in XN , where pNX is the probability mass function of PNX , defined by pNX (x) = PNX ({x}).This is essentially de Finetti’s (1937) definition for the exchangeability of a prevision. A useful relation existsbetween exchangeable coherent previsions and exchangeable coherent lower previsions:Proposition 1. Let PNX be the lower envelope of some set of coherent previsionsMNX . Then PNX is exchangeableif and only if all the coherent previsions PNX inMNX are exchangeable.Exchangeable coherent lower previsions have a very simple representation, in terms of sampling withoutreplacement. Consider any x ∈X N , then the so-called invariant atom [x] = {pix : pi ∈PN} is the smallestnon-empty subset of X N that contains x and that is invariant under all permutations pi in PN . We cancharacterise these invariant atoms using the counting maps TNx : XN → N0 defined for all x ∈X in such a waythat TNx (z) = TNx (z1, . . . ,zN) := |{k ∈ {1, . . . ,N} : zk = x}|, is the number of components of the N-tuple z thatassume the value x. We denote by TNX the vector-valued map fromXN to NX0 whose components are the TNx ,x∈X . Observe that TNX actually assumes values in the set of count vectorsN NX :={m ∈ NX0 : ∑x∈X mx = N}.Since permuting the components of a vector leaves the counts invariant, for all y and z in X N it holds thaty∈ [z] iff TNX (y) =TNX (z). If TNX (x) =m, then we denote [x] also by [m]. It has ν(m) :=(Nm)=N!/∏x∈X mx!elements.If the joint random variable X = (X1, . . . ,XN) assumes the value z in X N , then the correspondingcount vector assumes the value TNX (z) in NNX . This means that we can see TNX (X) = TNX (X1, . . . ,XN) asa random variable in N NX . If the available information about the values that X assumes in XN is givenby the coherent exchangeable lower prevision PNX , then the corresponding uncertainty model for the valuesthat TNX (X) assumes in NNX is given by the coherent induced lower prevision QNXon L (N NX ), given byQNX(h) := PNX (h◦TNX ) = PNX(∑m∈N NX h(m)I[m])for all gambles h onN NX . Any exchangeable coherent lowerprevision PNX is in fact completely determined by the corresponding distribution QNXof the count vectors.Theorem 2 (Representation theorem for exchangeable variables). Let N ≥ 1 and let PNX be a coherent exchange-able lower prevision onL (X N). Let f be any gamble onX N . Then the following statements hold:1. PNX ( f ) = PNX ( fˆ ) where fˆ is the gamble onXN that is constant on the permutation invariant atoms ofX N ,and given by fˆ (z) =MuHyNX ( f |m) := 1ν(m) ∑z∈[m] f (z) for all z ∈ [m], where m ∈N NX ;2. PNX ( f ) =QNX(MuHyNX ( f |·)), where MuHyNX ( f |·) is the gamble onN NX that takes the value MuHyNX ( f |m)in m ∈N NX .Consequently, a lower prevision on L (X N) is exchangeable if and only if it has the form Q(MuHyNX (·|·)),where Q is any coherent lower prevision onL (N NX ).AsMuHyNX ( f |m) is the prevision associated with the multiple hypergeometric distribution, this theorem impliesthat any collection of N exchangeable random variables inX can be seen as the result of N random drawingsfrom an urn with N balls whose types are characterised by the elements x of X , whose composition m isunknown, but for which the available information about the composition is modelled by a coherent lowerprevision onL (N NX ). That exchangeable coherent previsions can be interpreted in terms of sampling withoutreplacement from an urn with unknown composition, is actually well-known, and essentially goes back to deFinetti’s (1937) work on exchangeability. Our result for the more general case of exchangeable coherent lowerprevisions and random variables that may assume more than two values is a special case of a much more generalrepresentation theorem (De Cooman and Miranda, 2007).3 Exchangeable sequencesConsider now a sequence X1, . . . , Xn, . . . of random variables taking values in the same non-empty setX . Thenthis sequence is called exchangeable if any finite collection of random variables taken from this sequence isexchangeable. We can regard the exchangeable sequence as a single random variable X assuming values inthe setX N, where N is the set of the natural numbers. Its possible values x are sequences x1, . . . , xn, . . . ofelements ofX . We can model the available information about the value that X assumes inX N by a coherentlower prevision PNX onL (XN), called the distribution of the exchangeable random sequence X. The randomsequence X, or its distribution PNX , is exchangeable if and only if all itsXn-marginals PnX are exchangeablefor n≥ 1, where for any gamble f onX n it holds that PnX ( f ) = PNX ( f˜ ), where f˜ is the cylindrical extensionof f to X N: for all x ∈X N, f˜ (x) := f (x1, . . . ,xn). In addition, the family of exchangeable coherent lowerprevisions PnX , n≥ 1, satisfies the following ‘time consistency’ requirement: PnX ( f ) = Pn+kX ( f˜ ), for all n≥ 1,k≥ 0, and all gambles f onX n, where f˜ now denotes the cylindrical extension of f toX n+k: PnX should be theX n-marginal of any Pn+kX . In terms of the distributions of the count vectors, the time consistency requirementmeans that QnX(h) = Qn+kX(∑m∈N nXν(·−m)ν(m)ν(·) h(m))for all n≥ 1, k ≥ 0 and h ∈L (N nX ).Any collection of n≥ 1 random variables taken from such an exchangeable sequence has the same distri-bution as the first n variables X1, . . . , Xn, which is the exchangeable coherent lower prevision PnX onL (Xn).Conversely, suppose we have a collection of exchangeable coherent lower previsions PnX on L (Xn), n≥ 1that satisfy the time consistency requirement. Then any coherent lower prevision PNX on L (XN) that hasX n-marginals PnX is exchangeable. The smallest, or most conservative such (exchangeable) coherent lowerprevision is given by ENX ( f ) := supn∈NPnX(projn( f )), where f is any gamble onX N, and its lower projectionprojn( f ) onX n is the gamble onX n that is defined by projn( f )(x) := infzk=xk,k=1,...,n f (z) for all x ∈X n.De Finetti (1937 and 1974-1975) has proven a representation result for exchangeable sequences withcoherent previsions that generalises Theorem 2, and where multinomial distributions take over the rôle that themultiple hypergeometric ones play for finite collections of exchangeable variables. Here, we present another,arguably even simpler, way to prove the same results, which is moreover valid for coherent lower previsions.Consider a sequence of IID random variables Y1, . . . , Yn, with probability mass function θ , where θ is an elementof theX -simplex ΣX ={θ ∈ RX : (∀x ∈X )(θx ≥ 0) and ∑x∈X θx = 1}. The distribution of (Y1, . . . ,Yn) isMnnX ( f |θ ) = ∑z∈X N f (z)∏x∈X θTx(z)x = ∑m∈N NX ∑z∈[m] f (z)∏x∈X θmxx= ∑m∈N nX MuHyNX ( f |m)ν(m)∏x∈X θmxx =: CoMnnX(MuHynX ( f |·)∣∣θ ).It follows from Theorem 2 that the multinomial distribution MnnX (·|θ ) is exchangeable, and that CoMnnX (·|θ )is the corresponding distribution for the corresponding count vectors TnX (Y1, . . . ,Yn). Therefore the sequence ofIID random variables Y1, . . . , Yn, . . . is exchangeable.For any gamble f onX N , and as a function of θ ,MnNX ( f |θ ) is always a polynomial gamble on ΣX . Now,if RX is any coherent lower prevision on the linear space V (ΣX ) of all polynomial gambles on theX -simplex,then one can see that the family of coherent lower previsions PnX , n≥ 1, defined by PnX ( f ) = RX(MnnX ( f |·)),for any f ∈L (X n) is still exchangeable and time consistent, and the corresponding count distributions aregiven by QnX( f ) = RX(CoMnnX (g|·))for any g ∈L (N nX ). A converse result also holds:Theorem 3 (Representation theorem for exchangeable sequences). Given a time consistent family of exchange-able coherent lower previsions PnX on L (Xn), n ≥ 1, there is a unique coherent lower prevision RX onV (ΣX ) such that for all n≥ 1, all f ∈L (X n) and all g ∈L (N nX ) it holds that PnX ( f ) = RX (MnnX ( f |·))and QnX(g) = RX(CoMnnX (g|·)).The crucial step in proving this consists in recognising that any representing RX , if it exists, is only uniquelydetermined on V (ΣX ) ={CoMnnX (g|·) : n≥ 1,g ∈L (N nX )}, each of whose elements is a polynomial func-tion on theX -simplex. So the basic idea is to define RX on V (ΣX ) as follows: consider any element p ofV (ΣX ). Then, by definition, there are some n≥ 1 and g ∈L (N nX ) such that p= CoMnnX (g|·). We then letRX (p) := QnX(g). This definition is consistent, and the functional RX thus defined on the linear space VX isindeed a coherent lower prevision.Next, consider the sequence of so-called frequency random variables Fn := TnX (X1, . . . ,Xn)/n correspond-ing to an exchangeable sequence of random variables X1, . . . , Xn, . . . , and assuming values in theX -simplexΣX . The distribution PFn , i.e., the coherent lower prevision onL (ΣX ) that models the available informationabout the values that Fn assumes in ΣX , is given by PFN (h) = QnX(h ◦ 1n) = RX(CoMnnX (h ◦ 1n |·))for anyh ∈L (ΣX ). So we find the following result, which provides an interpretation for the representation RX , andwhich can be seen as another generalisation of de Finetti’s Representation Theorem.Theorem 4. For all continuous gambles h on ΣX , limn→∞PFn(h) = RX (h), and in this specific sense, thesample frequencies Fn converge in distribution.4 Extending local assessments to an exchangeable coherent lower previsionIn practice, a subject will usually make an assessment that N variables X1, . . . , XN in a setX are exchangeable,in addition to specifying supremum acceptable buying prices P( f ) for all gambles in some set of gamblesK ⊆L (X N). Can we turn these assessments into an exchangeable coherent lower prevision PNX defined onall ofL (X N), that is furthermore as small (least-committal, conservative) as possible?We start by symmetrising P. On the symmetrised domainKs := {pi f : f ∈K and pi ∈PN}, define thelower prevision Ps by letting Ps(pi f ) := P( f ) for all f ∈K and pi ∈PN . WhenK already contains gamblesthat are related to each other through (non-trivial) permutations, this may lead to conflicts, but then it makesgood sense to define Ps on such gambles by taking the highest value that P takes on them. In this way, we end upwith the point-wise smallest (most conservative) permutable (Walley, 1991) lower prevision Ps that dominates PonK . Its natural extension Es to all gambles is the point-wise smallest coherent extension of Ps to all gamblesonX N , and can be determined efficiently (Walley, 1991).Theorem 5. If Ps avoids sure loss, then there are exchangeable coherent lower previsions on L (X N) thatdominate P on its domainK . The count distribution QNXof the point-wise smallest such exchangeable coherentlower prevision PNX is given by QNX(h) = Es(∑m∈N NX h(m)I[m]), for all gambles g onNNX .REFERENCESA. P. Dawid. Probability, symmetry, and frequency. British J. Philos. Sci., 36(2):107–128, 1985.G. de Cooman and E. Miranda. Symmetry of models versus models of symmetry. In W. L. Harper and G. R. Wheeler,editors, Probability and Inference: Essays in Honor of Henry E. Kyburg, Jr. King’s College Publications, 2007.B. de Finetti. La prévision: ses lois logiques, ses sources subjectives. Ann. Inst. H. Poincaré, 7:1–68, 1937.B. de Finetti. Theory of Probability. John Wiley & Sons, Chichester, 1974–1975.P. Walley. Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, London, 1991.

Representing and assessing exchangeable lower previsions

https://biblio.ugent.be/publication/377955/file/579768.pdf

Representing and assessing exchangeable lower previsions

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography