This is the final version of the article. It first appeared from the Institute of Mathematical Statistics via http://dx.doi.org/10.1214/15-STS53

Arratia, Richard

Barbour, A. D.

Tavaré, Simon

English

Richard Arratia

A. D. Barbour

Simon Tavaré

Crossref

Statistical Science

Exploiting the Feller Coupling for the Ewens Sampling Formula

Barbour, A D

ZORA

Apollo (Cambridge)

Statistical Science2016, Vol. 31, No. 1, 27–29DOI: 10.1214/15-STS537Main article DOI: 10.1214/15-STS529© Institute of Mathematical Statistics, 2016Exploiting the Feller Coupling for theEwens Sampling FormulaRichard Arratia, A. D. Barbour and Simon TavaréWe congratulate Harry Crane on a masterful survey,showing the universal character of the Ewens samplingformula.There are two grand ways to get a simple handleon the Ewens sampling formula; one is the Chineserestaurant coupling, and the other is the Feller cou-pling. Since Crane has discussed the Chinese Restau-rant process, but not the Feller coupling, we will give abrief survey of the latter.The Ewens sampling formula, given in Crane’s (1),has an interpretation in terms of the cycle type of arandom permutation of n objects. For θ = 1, it is justCauchy’s formula, expressed in terms of the fraction ofpermutations of n objects that have exactly mi cyclesof order i, 1 ≤ i ≤ n. For general θ , the powerθm1+m2+···+mn = θKappearing in the formula, where K denotes the numberof cycles, biases the uniform random choice of a per-mutation by weighting with the factor θK , the remain-ing factors involving θ merely reflecting the new nor-malization constant required to specify a probabilitydistribution. We use the notation (C1(n), . . . ,Cn(n))to denote a random object distributed according to theEwens sampling formula, suppressing the parameterθ but making explicit the parameter n, so that, withCrane’s notation (1),P(C1(n) = m1, . . . ,Cn(n) = mn)(1)= p(m1, . . . ,mn; θ).Richard Arratia is Professor, Department of Mathematics,University of Southern California, 3620 S. Vermont Ave,KAP 104, Los Angeles, California 90089-2532, USA(e-mail: rarratia@usc.edu). A. D. Barbour is ProfessorEmeritus, Institut für Mathematik, Universität Zürich,Winterthurerstrasse 190, 8057 Zürich, Switzerland (e-mail:a.d.barbour@math.uzh.ch). Simon Tavaré is Professor,Department of Applied Mathematics and TheoreticalPhysics, University of Cambridge, Centre for MathematicalSciences, Wilberforce Road, Cambridge CB3 0WA, UnitedKingdom (e-mail: st321@cam.ac.uk).The Feller coupling, motivated by the example in Feller([6], page 815) is defined as follows. Take indepen-dent Bernoulli random variables ξi , i = 1,2,3, . . . ,with the simple odds ratios P(ξi = 0)/P(ξi = 1) =(i − 1)/θ . Thus, Eξi = P(ξi = 1) = θ/(θ + i − 1),and P(ξi = 0) = (i − 1)/(θ + i − 1). Say that an -spacing occurs in a sequence a1, a2, . . . , of zeros andones, starting at position i −  and ending at position i,if ai−ai−+1 · · ·ai−1ai = 10−11, a one followed by − 1 zeros followed by another one. Then if, for each ≥ 1, we defineC(n) := the number of -spacings inξ1, ξ2, . . . , ξn−1, ξn,1,0,0, . . . ,the joint distribution of C1(n), . . . ,Cn(n) is the Ewenssampling formula, as per Crane’s (1) and our (1). Thiscan be seen directly, for the case θ = 1: consider arandom permutation of 1 to n, write the canonicalcycle notation one symbol at a time, and let ξi indi-cate the decision to complete a cycle, when there isan i-way choice of which element to assign next. Thegeneral case θ > 0 follows by biasing, with respectto θK : since K = ξ1 + · · · + ξn, and the ξ1, . . . , ξnare independent, biasing their joint distribution byθξ1+···+ξn = θξ1 · · · θξn preserves their independenceand Bernoulli distributions, while changing the oddsP(ξi = 0)/P(ξi = 1) from (i − 1)/1 to (i − 1)/θ .Now, the wonderful thing that happens is that, withY defined to be the number of -spacings in the in-finite sequence ξ1, ξ2, . . . , it turns out that Y1, Y2, . . .are mutually independent, and that Y is Poisson dis-tributed, with EY = θ/, as in formula (11) in Sec-tion 3.8. This shows that the Ewens sampling formulais closely related to the simpler independent processY1, Y2, . . . , Yn. Explicitly, let Rn be the position ofthe rightmost one in ξ1, ξ2, . . . , ξn−1, ξn—noting thatalways ξ1 = 1 so Rn is well-defined—and let Jn :=(n + 1) − Rn. We haveC(n) ≤ Y + 1(Jn = ), 1 ≤  ≤ n,(2)with contributions to strict inequality whenever, forsome 1 ≤  ≤ n, an -spacing occurred in ξ1, ξ2, . . .starting at i −  and ending at i > n.2728 R. ARRATIA, A. D. BARBOUR AND S. TAVARÉWe view (2) as saying that the Ewens samplingformula distributed (C1(n), . . . ,Cn(n)) can be con-structed from the independent Poisson Y ’s using atmost one insertion, together with a random number ofdeletions. The expected number of deletions is Oθ(1),that is, bounded over all n, with the upper bound de-pending on the value of θ . A concrete upper boundis given in [3], but the limit value, call it c(θ), iscleaner. This limit is the expected number of spacingsof length at most 1, with right end greater than 1, in thescale invariant Poisson process on (0,∞) with inten-sity θ/x dx; see [1]. We havec(θ) =∫x>1θxP(at least one arrival in (x − 1, x))dx=∫x>1θx(1 − exp(−∫ xx−1θydy))dx=∫x>1θx(1 −(1 − 1x)θ)dxand, using the substitution v = 1 − 1/x, we getc(θ) = θ∫ 10(1 − v)−1(1 − vθ )dv= θ ∑n≥0(1n + 1 −1n + 1 + θ)= θ(1θ+ ∑n≥0(1n + 1 −1n + θ))= 1 + θ(γ + ψ(θ)),where γ is Euler’s constant and ψ is the digammafunction.The simple fact that one can transform the Ewenssampling formula into the highly tractable Poisson pro-cess Y1, Y2, . . . , Yn using a bounded (in expectation)number of insertions and deletions is, in itself, quitepowerful, since there are interesting aspects of the jointdistribution which are insensitive to a bounded numberof insertions and deletions. For example, consider theErdős–Turán law for the order of a random permuta-tion. The order of a permutation is the least commonmultiple of the lengths of its cycles, and the Erdős–Turán law is the statement of convergence to the stan-dard normal distribution, for the log of the order, cen-tered by subtracting an asymptotic mean log2 n/2, andscaling by dividing by an asymptotic standard devia-tion, log3/2 n/3. The effect of a finite number of cyclelengths is washed away by the scaling; see [5] for de-tails.In a similar spirit, and modeled after the Feller cou-pling for the Ewens sampling formula, [2] shows thatfor a random integer chosen uniformly from 1 to n,the counts Cp(n) of prime factors, including multi-plicity, can be coupled to independent Z2,Z3,Z5, . . .with P(Zp ≥ k) = p−k for prime p and k = 0,1,2, . . .in such a way that E∑p≤n |Cp(n) − Zp| ≤ 2 +O((log logn)2/ logn); informally, the prime factoriza-tion can be converted into the process of independentgeometric random variables, using on average no morethan 2 + εn insertions and deletions. The fact of beingable to convert with o(log logn) insertions and dele-tions already easily implies the Hardy–Ramanujan the-orem for the normal order of the number of primedivisors, and the fact of being able to convert witho(√log logn) insertions and deletions readily impliesthe Erdős–Kac central limit theorem for the number ofprime divisors.The Feller coupling expresses the Ewens samplingformula in terms of the spacings of the independentBernoulli sequence ξ1, ξ2, . . . , ξn. The conditioningrelation, described in Crane’s article at the start ofSection 3.8, expresses the Ewens sampling formulain terms of the independent Poisson Y1, Y2, . . . , Yn.Both these independent processes have the same limitupon rescaling, namely, the scale invariant Poissonprocess on (0,∞) with intensity θ/x dx. This leadsto a property of the scale invariant Poisson process:the set of its spacings has the same distribution asthe set of its arrivals. This property can be exploitedto bound the distance to the Poisson–Dirichlet limit,which is mentioned in Crane’s Section 4.2. Write(X1,X2, . . .) for the random vector distributed ac-cording to the Poisson–Dirichlet(θ = 1). For randompermutations, writing Li(n) for the size of the ithlargest cycle, [4] shows that there are couplings whichachieve E∑i≥1 |Li(n) − nXi | ∼ 14 logn, and that nocoupling can achieve a constant smaller than 1/4.For prime factorizations, writing Pi(n) for the ithlargest prime factor of a random integer distributeduniformly from 1 to n, [2] shows that there is a cou-pling of random integers to Poisson–Dirichlet such thatE∑i≥1 | logPi(n)−(logn)Xi | = O(log logn), and theconjecture that O(1) can be achieved remains open.REFERENCES[1] ARRATIA, R. (1998). On the central role of scale invari-ant Poisson processes on (0,∞). In Microsurveys in Dis-crete Probability (Princeton, NJ, 1997). DIMACS Ser. Dis-crete Math. Theoret. Comput. Sci. 41 21–41. Amer. Math. Soc.,Providence, RI. MR1630407DISCUSSION 29[2] ARRATIA, R. (2002). On the amount of dependence in theprime factorization of a uniform random integer. In Contempo-rary Combinatorics. Bolyai Soc. Math. Stud. 10 29–91. JánosBolyai Math. Soc., Budapest. MR1919568[3] ARRATIA, R., BARBOUR, A. D. and TAVARÉ, S. (1992).Poisson process approximations for the Ewens sampling for-mula. Ann. Appl. Probab. 2 519–535. MR1177897[4] ARRATIA, R., BARBOUR, A. D. and TAVARÉ, S. (2006).A tale of three couplings: Poisson–Dirichlet and GEM approx-imations for random permutations. Combin. Probab. Comput.15 31–62. MR2195574[5] BARBOUR, A. D. and TAVARÉ, S. (1994). A rate for theErdős–Turán law. Combin. Probab. Comput. 3 167–176.MR1288438[6] FELLER, W. (1945). The fundamental limit theorems in prob-ability. Bull. Amer. Math. Soc. (N.S.) 51 800–832. MR0013252

Exploiting the feller coupling for the ewens sampling formula

Exploiting the feller coupling for the ewens sampling formula

Abstract

Similar works

Full text

Available Versions

Crossref

ZORA

Apollo (Cambridge)