1,044 research outputs found
A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models
Peer reviewe
Imputation estimators for unnormalized models with missing data
Several statistical models are given in the form of unnormalized densities,
and calculation of the normalization constant is intractable. We propose
estimation methods for such unnormalized models with missing data. The key
concept is to combine imputation techniques with estimators for unnormalized
models including noise contrastive estimation and score matching. In addition,
we derive asymptotic distributions of the proposed estimators and construct
confidence intervals. Simulation results with truncated Gaussian graphical
models and the application to real data of wind direction reveal that the
proposed methods effectively enable statistical inference with unnormalized
models from missing data.Comment: To appear (AISTATS 2020
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
We propose a general algorithm for approximating nonstandard Bayesian
posterior distributions. The algorithm minimizes the Kullback-Leibler
divergence of an approximating distribution to the intractable posterior
distribution. Our method can be used to approximate any posterior distribution,
provided that it is given in closed form up to the proportionality constant.
The approximation can be any distribution in the exponential family or any
mixture of such distributions, which means that it can be made arbitrarily
precise. Several examples illustrate the speed and accuracy of our
approximation method in practice
Computationally Efficient Nonparametric Importance Sampling
The variance reduction established by importance sampling strongly depends on
the choice of the importance sampling distribution. A good choice is often hard
to achieve especially for high-dimensional integration problems. Nonparametric
estimation of the optimal importance sampling distribution (known as
nonparametric importance sampling) is a reasonable alternative to parametric
approaches.In this article nonparametric variants of both the self-normalized
and the unnormalized importance sampling estimator are proposed and
investigated. A common critique on nonparametric importance sampling is the
increased computational burden compared to parametric methods. We solve this
problem to a large degree by utilizing the linear blend frequency polygon
estimator instead of a kernel estimator. Mean square error convergence
properties are investigated leading to recommendations for the efficient
application of nonparametric importance sampling. Particularly, we show that
nonparametric importance sampling asymptotically attains optimal importance
sampling variance. The efficiency of nonparametric importance sampling
algorithms heavily relies on the computational efficiency of the employed
nonparametric estimator. The linear blend frequency polygon outperforms kernel
estimators in terms of certain criteria such as efficient sampling and
evaluation. Furthermore, it is compatible with the inversion method for sample
generation. This allows to combine our algorithms with other variance reduction
techniques such as stratified sampling. Empirical evidence for the usefulness
of the suggested algorithms is obtained by means of three benchmark integration
problems. As an application we estimate the distribution of the queue length of
a spam filter queueing system based on real data.Comment: 29 pages, 7 figure
Population annealing: Theory and application in spin glasses
Population annealing is an efficient sequential Monte Carlo algorithm for
simulating equilibrium states of systems with rough free energy landscapes. The
theory of population annealing is presented, and systematic and statistical
errors are discussed. The behavior of the algorithm is studied in the context
of large-scale simulations of the three-dimensional Ising spin glass and the
performance of the algorithm is compared to parallel tempering. It is found
that the two algorithms are similar in efficiency though with different
strengths and weaknesses.Comment: 16 pages, 10 figures, 4 table
- …