Black Box Variational Inference

Variational inference has become a widely used method to approximate posteriors in complex latent variables models. However, deriving a variational inference algorithm generally requires significant model-specific analysis, and these efforts can hinder and deter us from quickly developing and exploring a variety of models for a problem at hand. In this paper, we present a "black box" variational inference algorithm, one that can be quickly applied to many models with little additional derivation. Our method is based on a stochastic optimization of the variational objective where the noisy gradient is computed from Monte Carlo samples from the variational distribution. We develop a number of methods to reduce the variance of the gradient, always maintaining the criterion that we want to avoid difficult model-based derivations. We evaluate our method against the corresponding black box sampling based methods. We find that our method reaches better predictive likelihoods much faster than sampling methods. Finally, we demonstrate that Black Box Variational Inference lets us easily explore a wide space of models by quickly constructing and evaluating several models of longitudinal healthcare data

Variational Bayesian EM for SLAM

Designing accurate, robust and cost-effective systems is an important aspect of the research on self-driving vehicles. Radar is a common part of many existing automotive solutions and it is robust to adverse weather and lighting conditions, as such it can play an important role in the design of a self-driving vehicle. In this paper, a radar-based simultaneous localization and mapping (SLAM) algorithm using variational Bayesian expectation maximization (VBEM) is presented. The VBEM translates the inference problem to an optimization one. It provides an efficient and powerful method to estimate the unknown data association variables as well as the map of the environment as perceived by a radar and the unknown trajectory of the vehicle

Scalable Recommendation with Poisson Factorization

We develop a Bayesian Poisson matrix factorization model for forming recommendations from sparse user behavior data. These data are large user/item matrices where each user has provided feedback on only a small subset of items, either explicitly (e.g., through star ratings) or implicitly (e.g., through views or purchases). In contrast to traditional matrix factorization approaches, Poisson factorization implicitly models each user's limited attention to consume items. Moreover, because of the mathematical form of the Poisson likelihood, the model needs only to explicitly consider the observed entries in the matrix, leading to both scalable computation and good predictive performance. We develop a variational inference algorithm for approximate posterior inference that scales up to massive data sets. This is an efficient algorithm that iterates over the observed entries and adjusts an approximate posterior over the user/item representations. We apply our method to large real-world user data containing users rating movies, users listening to songs, and users reading scientific papers. In all these settings, Bayesian Poisson factorization outperforms state-of-the-art matrix factorization methods

The effect of missing data on robust Bayesian spectral analysis

This is the author accepted manuscript. The final version is available from the publisher via the DOI in this record.Published in: Machine Learning for Signal Processing (MLSP), 2013 IEEE International Workshop on Date of Conference: 22-25 Sept. 2013We investigate the effects of missing observations on the robust Bayesian model for spectral analysis introduced by Christmas [2013]. The model assumes Student-t distributed noise and uses an automatic relevance determination prior on the precisions of the amplitudes of the component sinusoids and it is not obvious what their effect will be when some of the otherwise temporally uniformly sampled data is missing

Variational Linear Response

A general linear response method for deriving improved estimates of correlations in the variational Bayes framework is presented. Three applications are given and it is discussed how to use linear response as a general principle for improving mean field approximations