22 research outputs found
Linear response methods for accurate covariance estimates from mean field variational bayes
Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, a well known failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable covariance. We generalize linear response methods from statistical physics to deliver accurate uncertainty estimates for model variables---both for individual variables and coherently across variables. We call our method linear response variational Bayes (LRVB). When the MFVB posterior approximation is in the exponential family, LRVB has a simple, analytic form, even for non-conjugate models. Indeed, we make no assumptions about the form of the true posterior. We demonstrate the accuracy and scalability of our method on a range of models for both simulated and real data
Fitting Structural Equation Models via Variational Approximations
Structural equation models are commonly used to capture the relationship
between sets of observed and unobservable variables. Traditionally these models
are fitted using frequentist approaches but recently researchers and
practitioners have developed increasing interest in Bayesian inference. In
Bayesian settings, inference for these models is typically performed via Markov
chain Monte Carlo methods, which may be computationally intensive for models
with a large number of manifest variables or complex structures. Variational
approximations can be a fast alternative; however, they have not been
adequately explored for this class of models. We develop a mean field
variational Bayes approach for fitting elemental structural equation models and
demonstrate how bootstrap can considerably improve the variational
approximation quality. We show that this variational approximation method can
provide reliable inference while being significantly faster than Markov chain
Monte Carlo