108 research outputs found
Discussion of "Functional Models for Time-Varying Random Objects'' by Dubey and M\"uller
The discussion focuses on metric covariance, a new association measure
between paired random objects in a metric space, developed by Dubey and
M\"uller, and on its relationship with other similar concepts which have
previously appeared in the literature, including distance covariance by
Sz\'ekely et al, as well as its generalisations which rely on the formalism of
reproducing kernel Hilbert spaces (RKHS)
K2-ABC: Approximate Bayesian Computation with Kernel Embeddings
Complicated generative models often result in a situation where computing the
likelihood of observed data is intractable, while simulating from the
conditional density given a parameter value is relatively easy. Approximate
Bayesian Computation (ABC) is a paradigm that enables simulation-based
posterior inference in such cases by measuring the similarity between simulated
and observed data in terms of a chosen set of summary statistics. However,
there is no general rule to construct sufficient summary statistics for complex
models. Insufficient summary statistics will "leak" information, which leads to
ABC algorithms yielding samples from an incorrect (partial) posterior. In this
paper, we propose a fully nonparametric ABC paradigm which circumvents the need
for manually selecting summary statistics. Our approach, K2-ABC, uses maximum
mean discrepancy (MMD) as a dissimilarity measure between the distributions
over observed and simulated data. MMD is easily estimated as the squared
difference between their empirical kernel embeddings. Experiments on a
simulated scenario and a real-world biological problem illustrate the
effectiveness of the proposed algorithm
A Note on Optimizing Distributions using Kernel Mean Embeddings
Kernel mean embeddings are a popular tool that consists in representing
probability measures by their infinite-dimensional mean embeddings in a
reproducing kernel Hilbert space. When the kernel is characteristic, mean
embeddings can be used to define a distance between probability measures, known
as the maximum mean discrepancy (MMD). A well-known advantage of mean
embeddings and MMD is their low computational cost and low sample complexity.
However, kernel mean embeddings have had limited applications to problems that
consist in optimizing distributions, due to the difficulty of characterizing
which Hilbert space vectors correspond to a probability distribution. In this
note, we propose to leverage the kernel sums-of-squares parameterization of
positive functions of Marteau-Ferey et al. [2020] to fit distributions in the
MMD geometry. First, we show that when the kernel is characteristic,
distributions with a kernel sum-of-squares density are dense. Then, we provide
algorithms to optimize such distributions in the finite-sample setting, which
we illustrate in a density fitting numerical experiment
Spread Divergences
For distributions p and q with different supports, the divergence D(p|q) may
not exist. We define a spread divergence on modified p and q and describe
sufficient conditions for the existence of such a divergence. We demonstrate
how to maximize the discriminatory power of a given divergence by
parameterizing and learning the spread. We also give examples of using a spread
divergence to train and improve implicit generative models, including linear
models (Independent Components Analysis) and non-linear models (Deep Generative
Networks)
Informative Features for Model Comparison
Given two candidate models, and a set of target observations, we address the
problem of measuring the relative goodness of fit of the two models. We propose
two new statistical tests which are nonparametric, computationally efficient
(runtime complexity is linear in the sample size), and interpretable. As a
unique advantage, our tests can produce a set of examples (informative
features) indicating the regions in the data domain where one model fits
significantly better than the other. In a real-world problem of comparing GAN
models, the test power of our new test matches that of the state-of-the-art
test of relative goodness of fit, while being one order of magnitude faster.Comment: Accepted to NIPS 201
- …