Control Variates for Reversible MCMC Samplers

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control variates, and we introduce a new, consistent estimator for the values of the coefficients of the optimal linear combination of these functions. The form and proposed construction of the control variates is derived from our solution of the Poisson equation associated with a specific MCMC scenario. The new estimator, which can be applied to the same MCMC sample, is derived from a novel, finite-dimensional, explicit representation for the optimal coefficients. The resulting variance-reduction methodology is primarily applicable when the simulated data are generated by a conjugate random-scan Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant, and that in some cases it can be quite dramatic. Extensions of this methodology in several directions are given, including certain families of Metropolis-Hastings samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding simulation examples are presented illustrating the utility of the proposed methods. All methodological and asymptotic arguments are rigorously justified under easily verifiable and essentially minimal conditions.Comment: 44 pages; 6 figures; 5 table

Exact sampling for intractable probability distributions via a Bernoulli factory

Many applications in the field of statistics require Markov chain Monte Carlo methods. Determining appropriate starting values and run lengths can be both analytically and empirically challenging. A desire to overcome these problems has led to the development of exact, or perfect, sampling algorithms which convert a Markov chain into an algorithm that produces i.i.d. samples from the stationary distribution. Unfortunately, very few of these algorithms have been developed for the distributions that arise in statistical applications, which typically have uncountable support. Here we study an exact sampling algorithm using a geometrically ergodic Markov chain on a general state space. Our work provides a significant reduction to the number of input draws necessary for the Bernoulli factory, which enables exact sampling via a rejection sampling approach. We illustrate the algorithm on a univariate Metropolis-Hastings sampler and a bivariate Gibbs sampler, which provide a proof of concept and insight into hyper-parameter selection. Finally, we illustrate the algorithm on a Bayesian version of the one-way random effects model with data from a styrene exposure study.Comment: 28 pages, 2 figure

Eigenspectrum bounds for semirandom matrices with modular and spatial structure for neural networks

The eigenvalue spectrum of the matrix of directed weights defining a neural network model is informative of several stability and dynamical properties of network activity. Existing results for eigenspectra of sparse asymmetric random matrices neglect spatial or other constraints in determining entries in these matrices, and so are of partial applicability to cortical-like architectures. Here we examine a parameterized class of networks that are defined by sparse connectivity, with connection weighting modulated by physical proximity (i.e., asymmetric Euclidean random matrices), modular network partitioning, and functional specificity within the excitatory population. We present a set of analytical constraints that apply to the eigenvalue spectra of associated weight matrices, highlighting the relationship between connectivity rules and classes of network dynamics

A Fast Algorithm for Sampling from the Posterior of a von Mises distribution

Motivated by molecular biology, there has been an upsurge of research activities in directional statistics in general and its Bayesian aspect in particular. The central distribution for the circular case is von Mises distribution which has two parameters (mean and concentration) akin to the univariate normal distribution. However, there has been a challenge to sample efficiently from the posterior distribution of the concentration parameter. We describe a novel, highly efficient algorithm to sample from the posterior distribution and fill this long-standing gap

Space-Time Signal Design for Multilevel Polar Coding in Slow Fading Broadcast Channels

Slow fading broadcast channels can model a wide range of applications in wireless networks. Due to delay requirements and the unavailability of the channel state information at the transmitter (CSIT), these channels for many applications are non-ergodic. The appropriate measure for designing signals in non-ergodic channels is the outage probability. In this paper, we provide a method to optimize STBCs based on the outage probability at moderate SNRs. Multilevel polar coded-modulation is a new class of coded-modulation techniques that benefits from low complexity decoders and simple rate matching. In this paper, we derive the outage optimality condition for multistage decoding and propose a rule for determining component code rates. We also derive an upper bound on the outage probability of STBCs for designing the set-partitioning-based labelling. Finally, due to the optimality of the outage-minimized STBCs for long codes, we introduce a novel method for the joint optimization of short-to-moderate length polar codes and STBCs

Random variate generation and connected computational issues for the Poisson–Tweedie distribution

After providing a systematic outline of the stochastic genesis of the Poisson–Tweedie distribution, some computational issues are considered. More specifically, we introduce a closed form for the probability function, as well as its corresponding integral representation which may be useful for large argument values. Several algorithms for generating Poisson–Tweedie random variates are also suggested. Finally, count data connected to the citation profiles of two statistical journals are modeled and analyzed by means of the Poisson–Tweedie distribution