Improved Scaling with Dimension in the Bernstein-von Mises Theorem for Two Statistical Models

Past works have shown that the Bernstein-von Mises theorem, on the asymptotic normality of posterior distributions, holds if the parameter dimension $d$ grows slower than the cube root of sample size $n$. Here, we prove the first Bernstein-von Mises theorem in the regime $d^2\ll n$. We establish this result for 1) exponential families and 2) logistic regression with Gaussian design. The proof builds on our recent work on the accuracy of the Laplace approximation to posterior distributions, in which we showed the approximation error in TV distance scales as $d/\sqrt n$

Tight Dimension Dependence of the Laplace Approximation

In Bayesian inference, a widespread technique to approximately sample from and compute statistics of a high-dimensional posterior is to use the Laplace approximation, a Gaussian proxy to the posterior. The Laplace approximation accuracy improves as sample size grows, but the question of how fast dimension $d$ can grow with sample size $n$ has not been fully resolved. Prior works have shown that $d^3\ll n$ is a sufficient condition for accuracy of the approximation. But by deriving the leading order contribution to the TV error, we show that $d^2\ll n$ is sufficient. We show for a logistic regression posterior that this growth condition is necessary

Tight skew adjustment to the Laplace approximation in high dimensions

In Bayesian inference, a simple and popular approach to reduce the burden of computing high dimensional integrals against a posterior $\pi$ is to make the Laplace approximation $\hat\gamma$. This is a Gaussian distribution, so computing $\int fd\pi$ via the approximation $\int fd\hat\gamma$ is significantly less expensive. In this paper, we make two general contributions to the topic of high-dimensional Laplace approximations, as well as a third contribution specific to a logistic regression model. First, we tighten the dimension dependence of the error $|\int fd\pi - \int fd\hat\gamma|$ for a broad class of functions $f$. Second, we derive a higher-accuracy approximation $\hat\gamma_S$ to $\pi$, which is a skew-adjusted modification to $\hat\gamma$. Our third contribution - in the setting of Bayesian inference for logistic regression with Gaussian design - is to use the first two results to derive upper bounds which hold uniformly over different sample realizations, and lower bounds on the Laplace mean approximation error. In particular, we prove a skewed Bernstein-von Mises Theorem in this logistic regression setting

On De Graaf spaces of pseudoquotients

A space of pseudoquotients, $\mathcal{B}(X,S)$, is defined as equivalence classes of pairs $(x,f)$, where $x$ is an element of a non-empty set $X$, $f$ is an element of $S$, a commutative semigroup of injective maps from $X$ to $X$, and $(x,f) \sim (y,g)$ if $gx=fy$. In this note, we consider a generalization of this construction where the assumption of commutativity of $S$ is replaced by Ore type conditions. As in the commutative case, $X$ can be identified with a subset of $\mathcal{B}(X,S)$, and $S$ can be extended to a group, $G$, of bijections on $\mathcal{B}(X,S)$. We introduce a natural topology on $\mathcal{B}(X,S)$ and show that all elements of $G$ are homeomorphisms on $\mathcal{B}(X,S)$

The hydrodynamic limit of a crystal surface jump diffusion with Metropolis-type rates

Crystal surface diffusion refers to the way in which atoms on the surface of a crystal redistribute to eventually settle into a configuration with minimal surface energy. Along with epitaxy, or crystal growth, crystal surface diffusion is important to study due to the role it plays in the production of thin films, which have wide-ranging applications in microelectronics. For example, the deformation of a crystal surface to an equilibrium state plays a central role in fuel cells that rely on thin crystal films, as the conversion efficiency of chemical energy to electricity depends on the surface configuration of the film. As is characteristic of large microscopic systems, we can gain more insight into the nature of the dynamics of surface diffusion by studying it at the macroscopic level than at the level of individual atoms. Since the physical process is microscopic, however, a faithful mathematical model of the diffusion should describe it with microscopic dynamics. Given a model of the microscopic dynamics, then, we are presented with the challenge of deriving macroscopic dynamics in the limit as the number of particles approaches infinity. This is known as a hydrodynamic, or scaling, limit; it is particularly appealing from the modeling perspective because the input is the true, microscopic dynamics, while the output is a much easier to analyze continuum equation. The main goal of this work is to derive such a scaling limit for a specific dynamics governing the microscopic process of crystal surface diffusion.Bachelor of Scienc

Analysis of a fourth order exponential PDE arising from a crystal surface jump process with Metropolis-type transition rates

We analytically and numerically study a fourth order PDE modeling rough crystal surface diffusion on the macroscopic level. We discuss existence of solutions globally in time and long time dynamics for the PDE model. The PDE, originally derived by the second author, is the continuum limit of a microscopic model of the surface dynamics, given by a Markov jump process with Metropolis type transition rates. We outline the convergence argument, which depends on a simplifying assumption on the local equilibrium measure that is valid in the high temperature regime. We provide numerical evidence for the convergence of the microscopic model to the PDE in this regime.Comment: 14 pages, 4 figures, comments welcome! Revised significantly thanks to very thorough referee reports. Some previous discussions have been removed and will be reported in a separate result by one of the author

Order In Spaces Of Pseudoquotients

We investigate properties of pseudoquotient extensions of sets with relations to ordered spaces, in particular. In the case of ordered spaces, we compare the standard quotient topology in the space of pseudoquotients with the order topology. Copyright © 2014 by Topology Proceedings

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

We derive an asymptotic expansion for the log-likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest.ISSN:0010-3640ISSN:1097-031