Iterative Updating of Model Error for Bayesian Inversion

In computational inverse problems, it is common that a detailed and accurate forward model is approximated by a computationally less challenging substitute. The model reduction may be necessary to meet constraints in computing time when optimization algorithms are used to find a single estimate, or to speed up Markov chain Monte Carlo (MCMC) calculations in the Bayesian framework. The use of an approximate model introduces a discrepancy, or modeling error, that may have a detrimental effect on the solution of the ill-posed inverse problem, or it may severely distort the estimate of the posterior distribution. In the Bayesian paradigm, the modeling error can be considered as a random variable, and by using an estimate of the probability distribution of the unknown, one may estimate the probability distribution of the modeling error and incorporate it into the inversion. We introduce an algorithm which iterates this idea to update the distribution of the model error, leading to a sequence of posterior distributions that are demonstrated empirically to capture the underlying truth with increasing accuracy. Since the algorithm is not based on rejections, it requires only limited full model evaluations. We show analytically that, in the linear Gaussian case, the algorithm converges geometrically fast with respect to the number of iterations. For more general models, we introduce particle approximations of the iteratively generated sequence of distributions; we also prove that each element of the sequence converges in the large particle limit. We show numerically that, as in the linear case, rapid convergence occurs with respect to the number of iterations. Additionally, we show through computed examples that point estimates obtained from this iterative algorithm are superior to those obtained by neglecting the model error.Comment: 39 pages, 9 figure

Bayesian peak-bagging of solar-like oscillators using MCMC: A comprehensive guide

Context: Asteroseismology has entered a new era with the advent of the NASA Kepler mission. Long and continuous photometric observations of unprecedented quality are now available which have stimulated the development of a number of suites of innovative analysis tools. Aims: The power spectra of solar-like oscillations are an inexhaustible source of information on stellar structure and evolution. Robust methods are hence needed in order to infer both individual oscillation mode parameters and parameters describing non-resonant features, thus making a seismic interpretation possible. Methods: We present a comprehensive guide to the implementation of a Bayesian peak-bagging tool that employs a Markov chain Monte Carlo (MCMC). Besides making it possible to incorporate relevant prior information through Bayes' theorem, this tool also allows one to obtain the marginal probability density function for each of the fitted parameters. We apply this tool to a couple of recent asteroseismic data sets, namely, to CoRoT observations of HD 49933 and to ground-based observations made during a campaign devoted to Procyon. Results: The developed method performs remarkably well at constraining not only in the traditional case of extracting oscillation frequencies, but also when pushing the limit where traditional methods have difficulties. Moreover it provides an rigorous way of comparing competing models, such as the ridge identifications, against the asteroseismic data.Comment: Accepted for publication in A&

Cosmology of neutrinos and extra light particles after WMAP3

We study how present data probe standard and non-standard properties of neutrinos and the possible existence of new light particles, freely-streaming or interacting, among themselves or with neutrinos. Our results include: sum m_nu < 0.40 eV at 99.9% C.L.; that extra massless particles have abundance Delta N_nu = 2 pm 1 if freely-streaming and Delta N_nu = 0 pm 1.3 if interacting; that 3 interacting neutrinos are disfavored at about 4 sigma. We investigate the robustness of our results by fitting to different sub-sets of data. We developed our own cosmological computational tools, somewhat different from the standard ones.Comment: 18 pages, 8 figures. Added in v2: an explicit comparison of our code with CAMB, some clarifications on the statistical analysis and some references. Matches version published in JCA

An optimally concentrated Gabor transform for localized time-frequency components

Gabor analysis is one of the most common instances of time-frequency signal analysis. Choosing a suitable window for the Gabor transform of a signal is often a challenge for practical applications, in particular in audio signal processing. Many time-frequency (TF) patterns of different shapes may be present in a signal and they can not all be sparsely represented in the same spectrogram. We propose several algorithms, which provide optimal windows for a user-selected TF pattern with respect to different concentration criteria. We base our optimization algorithm on $l^p$-norms as measure of TF spreading. For a given number of sampling points in the TF plane we also propose optimal lattices to be used with the obtained windows. We illustrate the potentiality of the method on selected numerical examples

MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmm R Package

Generalized linear mixed models provide a flexible framework for modeling a range of data, although with non-Gaussian response variables the likelihood cannot be obtained in closed form. Markov chain Monte Carlo methods solve this problem by sampling from a series of simpler conditional distributions that can be evaluated. The R package MCMCglmm implements such an algorithm for a range of model fitting problems. More than one response variable can be analyzed simultaneously, and these variables are allowed to follow Gaussian, Poisson, multi(bi)nominal, exponential, zero-inflated and censored distributions. A range of variance structures are permitted for the random effects, including interactions with categorical or continuous variables (i.e., random regression), and more complicated variance structures that arise through shared ancestry, either through a pedigree or through a phylogeny. Missing values are permitted in the response variable(s) and data can be known up to some level of measurement error as in meta-analysis. All simu- lation is done in C/ C++ using the CSparse library for sparse linear systems.

Conjugate Projective Limits

We characterize conjugate nonparametric Bayesian models as projective limits of conjugate, finite-dimensional Bayesian models. In particular, we identify a large class of nonparametric models representable as infinite-dimensional analogues of exponential family distributions and their canonical conjugate priors. This class contains most models studied in the literature, including Dirichlet processes and Gaussian process regression models. To derive these results, we introduce a representation of infinite-dimensional Bayesian models by projective limits of regular conditional probabilities. We show under which conditions the nonparametric model itself, its sufficient statistics, and -- if they exist -- conjugate updates of the posterior are projective limits of their respective finite-dimensional counterparts. We illustrate our results both by application to existing nonparametric models and by construction of a model on infinite permutations.Comment: 49 pages; improved version: revised proof of theorem 3 (results unchanged), discussion added, exposition revise

Max-Sliced Wasserstein Distance and its use for GANs

Generative adversarial nets (GANs) and variational auto-encoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, image-to-image translation and feature learning. However, to model high-dimensional distributions, sequential training and stacked architectures are common, increasing the number of tunable hyper-parameters as well as the training time. Nonetheless, the sample complexity of the distance metrics remains one of the factors affecting GAN training. We first show that the recently proposed sliced Wasserstein distance has compelling sample complexity properties when compared to the Wasserstein distance. To further improve the sliced Wasserstein distance we then analyze its `projection complexity' and develop the max-sliced Wasserstein distance which enjoys compelling sample complexity while reducing projection complexity, albeit necessitating a max estimation. We finally illustrate that the proposed distance trains GANs on high-dimensional images up to a resolution of 256x256 easily.Comment: Accepted to CVPR 201