Patterns of Scalable Bayesian Inference

Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assumptions about model structure, underlying computational resources, and the importance of asymptotic correctness. As a result, there is a zoo of ideas with few clear overarching principles. In this paper, we seek to identify unifying principles, patterns, and intuitions for scaling Bayesian inference. We review existing work on utilizing modern computing resources with both MCMC and variational approximation techniques. From this taxonomy of ideas, we characterize the general principles that have proven successful for designing scalable inference procedures and comment on the path forward

Markovian Dynamics on Complex Reaction Networks

Complex networks, comprised of individual elements that interact with each other through reaction channels, are ubiquitous across many scientific and engineering disciplines. Examples include biochemical, pharmacokinetic, epidemiological, ecological, social, neural, and multi-agent networks. A common approach to modeling such networks is by a master equation that governs the dynamic evolution of the joint probability mass function of the underling population process and naturally leads to Markovian dynamics for such process. Due however to the nonlinear nature of most reactions, the computation and analysis of the resulting stochastic population dynamics is a difficult task. This review article provides a coherent and comprehensive coverage of recently developed approaches and methods to tackle this problem. After reviewing a general framework for modeling Markovian reaction networks and giving specific examples, the authors present numerical and computational techniques capable of evaluating or approximating the solution of the master equation, discuss a recently developed approach for studying the stationary behavior of Markovian reaction networks using a potential energy landscape perspective, and provide an introduction to the emerging theory of thermodynamic analysis of such networks. Three representative problems of opinion formation, transcription regulation, and neural network dynamics are used as illustrative examples.Comment: 52 pages, 11 figures, for freely available MATLAB software, see http://www.cis.jhu.edu/~goutsias/CSS%20lab/software.htm

New factor graph based multiuser detector for spectrally efficient CPM

This paper presents a new iterative multiuser detection algorithm for asynchronous spectrally-efficient M-ary continuous-phase modulation in additive white Gaussian noise. This detection algorithm is closely related to another algorithm that was recently proposed by the same authors, but it follows from applying the sum-product algorithm to a different factor graph of the same multiuser detection problem. This, in turn, results in a different way to approximate the marginal bit a-posteriori probabilities that are used to perform minimum bit error rate multiuser detection. The girth of the factor graph considered in this contribution is twice as large, which is known to be potentially beneficial for the accuracy of the a-posteriori probabilities. The size of the largest factor graph variable alphabets also multiplies with M, rendering the straightforward application of the sum-product algorithm more complex. Through approximating a suitable set of sum-product messages by a Gaussian distribution, this complexity is significantly reduced. For a set of system parameters yielding high spectral efficiency, the resulting algorithm significantly outperforms the previously proposed solution