On Measure Concentration of Random Maximum A-Posteriori Perturbations

The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More efficient algorithms use sequential sampling strategies based on the expected value of low dimensional MAP perturbations. This paper develops new measure concentration inequalities that bound the number of samples needed to estimate such expected values. Applying the general result to MAP perturbations can yield a more efficient algorithm to approximate sampling from the Gibbs distribution. The measure concentration result is of general interest and may be applicable to other areas involving expected estimations

On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations

In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions. Specifically, we provide means for drawing either approximate or unbiased samples from Gibbs' distributions by introducing low dimensional perturbations and solving the corresponding MAP assignments. Our approach also leads to new ways to derive lower bounds on partition functions. We demonstrate empirically that our method excels in the typical "high signal - high coupling" regime. The setting results in ragged energy landscapes that are challenging for alternative approaches to sampling and/or lower bounds

Well-posedness of Bayesian inverse problems in quasi-Banach spaces with stable priors

The Bayesian perspective on inverse problems has attracted much mathematical attention in recent years. Particular attention has been paid to Bayesian inverse problems (BIPs) in which the parameter to be inferred lies in an infinite-dimensional space, a typical example being a scalar or tensor field coupled to some observed data via an ODE or PDE. This article gives an introduction to the framework of well-posed BIPs in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451--559, 2010) and others. This framework has the advantage of ensuring uniformly well-posed inference problems independently of the finite-dimensional discretisation used for numerical solution. Recently, this framework has been extended to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen--Lo\`eve expansion for square-integrable random variables can be used to sample such measures on quasi-Banach spaces. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger and total variation metrics upon perturbations of the misfit function and observed data.Comment: To appear in the proceedings of the 88th Annual Meeting of the International Association of Applied Mathematics and Mechanics (GAMM), Weimar 2017. This preprint differs from the final published version in pagination and typographical detai