9,519 research outputs found
Statistical Inference for Generative Models with Maximum Mean Discrepancy
While likelihood-based inference and its variants provide a statistically
efficient and widely applicable approach to parametric inference, their
application to models involving intractable likelihoods poses challenges. In
this work, we study a class of minimum distance estimators for intractable
generative models, that is, statistical models for which the likelihood is
intractable, but simulation is cheap. The distance considered, maximum mean
discrepancy (MMD), is defined through the embedding of probability measures
into a reproducing kernel Hilbert space. We study the theoretical properties of
these estimators, showing that they are consistent, asymptotically normal and
robust to model misspecification. A main advantage of these estimators is the
flexibility offered by the choice of kernel, which can be used to trade-off
statistical efficiency and robustness. On the algorithmic side, we study the
geometry induced by MMD on the parameter space and use this to introduce a
novel natural gradient descent-like algorithm for efficient implementation of
these estimators. We illustrate the relevance of our theoretical results on
several classes of models including a discrete-time latent Markov process and
two multivariate stochastic differential equation models
Statistical inference for generative models with maximum mean discrepancy
While likelihood-based inference and its variants provide a statistically efficient and widely applicable approach to parametric inference, their application to models involving intractable likelihoods poses challenges. In this work, we study a class of minimum distance estimators for intractable generative models, that is, statistical models for which the likelihood is intractable, but simulation is cheap. The distance considered, maximum mean discrepancy (MMD), is defined through the embedding of probability measures into a reproducing kernel Hilbert space. We study the theoretical properties of these estimators, showing that they are consistent, asymptotically normal and robust to model misspecification. A main advantage of these estimators is the flexibility offered by the choice of kernel, which can be used to trade-off statistical efficiency and robustness. On the algorithmic side, we study the geometry induced by MMD on the parameter space and use this to introduce a novel natural gradient descent-like algorithm for efficient implementation of these estimators. We illustrate the relevance of our theoretical results on several classes of models including a discrete-time latent Markov process and two multivariate stochastic differential equation models
AReS and MaRS - Adversarial and MMD-Minimizing Regression for SDEs
Stochastic differential equations are an important modeling class in many
disciplines. Consequently, there exist many methods relying on various
discretization and numerical integration schemes. In this paper, we propose a
novel, probabilistic model for estimating the drift and diffusion given noisy
observations of the underlying stochastic system. Using state-of-the-art
adversarial and moment matching inference techniques, we avoid the
discretization schemes of classical approaches. This leads to significant
improvements in parameter accuracy and robustness given random initial guesses.
On four established benchmark systems, we compare the performance of our
algorithms to state-of-the-art solutions based on extended Kalman filtering and
Gaussian processes.Comment: Published at the Thirty-sixth International Conference on Machine
Learning (ICML 2019
- …