Posterior Simulation in the Generalized Linear Model with Semiparmetric Random Effects

Generalized linear mixed models with semiparametric random effects are useful in a wide variety of Bayesian applications. When the random effects arise from a mixture of Dirichlet process (MDP) model, normal base measures and Gibbs sampling procedures based on the Pólya urn scheme are often used to simulate posterior draws. These algorithms are applicable in the conjugate case when (for a normal base measure) the likelihood is normal. In the non-conjugate case, the algorithms proposed by MacEachern and Müller (1998) and Neal (2000) are often applied to generate posterior samples. Some common problems associated with simulation algorithms for non-conjugate MDP models include convergence and mixing difficulties. This paper proposes an algorithm based on the Pólya urn scheme that extends the Gibbs sampling algorithms to non-conjugate models with normal base measures and exponential family likelihoods. The algorithm proceeds by making Laplace approximations to the likelihood function, thereby reducing the procedure to that of conjugate normal MDP models. To ensure the validity of the stationary distribution in the non-conjugate case, the proposals are accepted or rejected by a Metropolis-Hastings step. In the special case where the data are normally distributed, the algorithm is identical to the Gibbs sampler

Origins of the Combinatorial Basis of Entropy

The combinatorial basis of entropy, given by Boltzmann, can be written $H = N^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy, $N$ is the number of entities and $\mathbb{W}$ is number of ways in which a given realization of a system can occur (its statistical weight). This can be broadened to give generalized combinatorial (or probabilistic) definitions of entropy and cross-entropy: $H=\kappa (\phi(\mathbb{W}) +C)$ and $D=-\kappa (\phi(\mathbb{P}) +C)$, where $\mathbb{P}$ is the probability of a given realization, $\phi$ is a convenient transformation function, $\kappa$ is a scaling parameter and $C$ an arbitrary constant. If $\mathbb{W}$ or $\mathbb{P}$ satisfy the multinomial weight or distribution, then using $\phi(\cdot)=\ln(\cdot)$ and $\kappa=N^{-1}$, $H$ and $D$ asymptotically converge to the Shannon and Kullback-Leibler functions. In general, however, $\mathbb{W}$ or $\mathbb{P}$ need not be multinomial, nor may they approach an asymptotic limit. In such cases, the entropy or cross-entropy function can be {\it defined} so that its extremization ("MaxEnt'' or "MinXEnt"), subject to the constraints, gives the ``most probable'' (``MaxProb'') realization of the system. This gives a probabilistic basis for MaxEnt and MinXEnt, independent of any information-theoretic justification. This work examines the origins of the governing distribution $\mathbb{P}$.... (truncated)Comment: MaxEnt07 manuscript, version 4 revise

AN URN MODEL FOR CASCADING FAILURES ON A LATTICE

A cascading failure is a failure in a system of interconnected parts, in which the breakdown of one element can lead to the subsequent collapse of the others. The aim of this paper is to introduce a simple combinatorial model for the study of cascading failures. In particular, having in mind particle systems and Markov random fields, we take into consideration a network of interacting urns displaced over a lattice. Every urn is Pólya-like and its reinforcement matrix is not only a function of time (time contagion) but also of the behavior of the neighboring urns (spatial contagion), and of a random component, which can represent either simple fate or the impact of exogenous factors. In this way a non-trivial dependence structure among the urns is built, and it is used to study default avalanches over the lattice. Thanks to its flexibility and its interesting probabilistic properties, the given construction may be used to model different phenomena characterized by cascading failures such as power grids and financial network

Predictive Constructions Based on Measure-Valued Pólya Urn Processes

Measure-valued Pólya urn processes (MVPP) are Markov chains with an additive structure that serve as an extension of the generalized k-color Pólya urn model towards a continuum of pos- sible colors. We prove that, for any MVPP $(\mu_n)_{n ≥ 0}$ on a Polish space $\mathbb{X}$, the normalized sequence $( \mu_n / \mu_n (\mathbb{X}) )_{n \ge 0}$ agrees with the marginal predictive distributions of some random process $(X_n)_{n \ge 1}$. Moreover, $\mu_n = \mu_{n − 1} + R_{X_n}, \ n \ge 1$, where $x \mapsto R_x$ is a random transition kernel on $\mathbb{X}$; thus, if $\mu_{n − 1}$ represents the contents of an urn, then X n denotes the color of the ball drawn with distribution $\mu_{n − 1} / \mu_{n − 1}(\mathbb{X})$ and $R_{X_{n}}$ - the subsequent reinforcement. In the case $R_{X_{n}} = W_n\delta_{X_n}$, for some non-negative random weights $W_1, \ W_2, \$ ... , the process $( X_n )_{n \ge 1}$ is better understood as a randomly reinforced extension of Blackwell and MacQueen’s Pólya sequence. We study the asymptotic properties of the predictive distributions and the empirical frequencies of $( X_n )_{n \ge 1}$ under different assumptions on the weights. We also investigate a generalization of the above models via a randomization of the law of the reinforcement

Multiple drawing multi-colour urns by stochastic approximation

A classical Pólya urn scheme is a Markov process where the evolution is encoded by a replacement matrix (Ri, j)1 ≤ i, j ≤ d. At every discrete time-step, we draw a ball uniformly at random, denote its colour c, and replace it in the urn together with Rc, j balls of colour j (for all 1 ≤ j ≤ d). We study multiple drawing Pólya urns, where the replacement rule depends on the random drawing of a set of m balls from the urn (with or without replacement). Many particular examples of this situation have been studied in the literature, but the only general results are due to Kuba and Mahmoud (2017). These authors proved second-order asymptotic results in the two-colour case, under the so-called balance and affinity assumptions, the latter being somewhat artificial. The main idea of this work is to apply stochastic approximation methods to this problem, which enables us to prove analogous results to Kuba and Mahmoud, but without the artificial affinity hypothesis, and, for the first time in the literature, in the d-colour case (d ≥ 3). We also provide some partial results in the two-colour nonbalanced case, the novelty here being that the only results for this case currently in the literature are for particular examples.</p

Analyzing Generalized P\'olya Urn Models using Martingales, with an Application to Viral Evolution

The randomized play-the-winner (RPW) model is a generalized P\'olya Urn process with broad applications ranging from clinical trials to molecular evolution. We derive an exact expression for the variance of the RPW model by transforming the P\'olya Urn process into a martingale, correcting an earlier result of Matthews and Rosenberger (1997). We then use this result to approximate the full probability mass function of the RPW model for certain parameter values relevant to genetic applications. Finally, we fit our model to genomic sequencing data of SARS-CoV-2, demonstrating a novel method of estimating the viral mutation rate that delivers comparable results to existing scientific literature.Comment: 27 pages, 2 figure