Patterns of Scalable Bayesian Inference

Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assumptions about model structure, underlying computational resources, and the importance of asymptotic correctness. As a result, there is a zoo of ideas with few clear overarching principles. In this paper, we seek to identify unifying principles, patterns, and intuitions for scaling Bayesian inference. We review existing work on utilizing modern computing resources with both MCMC and variational approximation techniques. From this taxonomy of ideas, we characterize the general principles that have proven successful for designing scalable inference procedures and comment on the path forward

Integral, mean and covariance of the simplex-truncated multivariate normal distribution

Compositional data, which is data consisting of fractions or probabilities, is common in many fields including ecology, economics, physical science and political science. If these data would otherwise be normally distributed, their spread can be conveniently represented by a multivariate normal distribution truncated to the non-negative space under a unit simplex. Here this distribution is called the simplex-truncated multivariate normal distribution. For calculations on truncated distributions, it is often useful to obtain rapid estimates of their integral, mean and covariance; these quantities characterising the truncated distribution will generally possess different values to the corresponding non-truncated distribution. In this paper, three different approaches that can estimate the integral, mean and covariance of any simplex-truncated multivariate normal distribution are described and compared. These three approaches are (1) naive rejection sampling, (2) a method described by Gessner et al. that unifies subset simulation and the Holmes-Diaconis-Ross algorithm with an analytical version of elliptical slice sampling, and (3) a semi-analytical method that expresses the integral, mean and covariance in terms of integrals of hyperrectangularly-truncated multivariate normal distributions, the latter of which are readily computed in modern mathematical and statistical packages. Strong agreement is demonstrated between all three approaches, but the most computationally efficient approach depends strongly both on implementation details and the dimension of the simplex-truncated multivariate normal distribution. For computations in low-dimensional distributions, the semi-analytical method is fast and thus should be considered. As the dimension increases, the Gessner et al. method becomes the only practically efficient approach of the methods tested here

Discriminating between a Stochastic Gravitational Wave Background and Instrument Noise

The detection of a stochastic background of gravitational waves could significantly impact our understanding of the physical processes that shaped the early Universe. The challenge lies in separating the cosmological signal from other stochastic processes such as instrument noise and astrophysical foregrounds. One approach is to build two or more detectors and cross correlate their output, thereby enhancing the common gravitational wave signal relative to the uncorrelated instrument noise. When only one detector is available, as will likely be the case with the Laser Interferometer Space Antenna (LISA), alternative analysis techniques must be developed. Here we show that models of the noise and signal transfer functions can be used to tease apart the gravitational and instrument noise contributions. We discuss the role of gravitational wave insensitive "null channels" formed from particular combinations of the time delay interferometry, and derive a new combination that maintains this insensitivity for unequal arm length detectors. We show that, in the absence of astrophysical foregrounds, LISA could detect signals with energy densities as low as $\Omega_{\rm gw} = 6 \times 10^{-13}$ with just one month of data. We describe an end-to-end Bayesian analysis pipeline that is able to search for, characterize and assign confidence levels for the detection of a stochastic gravitational wave background, and demonstrate the effectiveness of this approach using simulated data from the third round of Mock LISA Data Challenges.Comment: 10 Pages, 10 Figure