Consistent Kernel Mean Estimation for Functions of Random Variables

We provide a theoretical foundation for non-parametrically estimating functions of random variables using kernel mean embeddings. We show that for any continuous function f, consistent estimators of the mean embedding of a random variable X lead to consistent estimators of the mean embedding of f(X). For Gaussian kernels and sufficiently smooth functions we also provide rates of convergence. Our results also apply for functions of multiple random variables. If the variables are dependent, we require an estimator of the mean embedding of their joint distribution as a starting point; if they are independent, it is sufficient to have separate mean embeddings of their marginal distributions. In either case, our results cover both mean embeddings expressed based on i.i.d. samples as well as reduced set expansions in terms of dependent expansion points. The latter serves as a justification for using such expansions to limit memory resources when we use the approach as a basis for probabilistic programming.Carl-Johann Simon-Gabriel is supported by a Google European Fellowship in Causal Inference

Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations

We describe a method to perform functional operations on probability distributions of random variables. The method uses reproducing kernel Hilbert space representations of probability distributions, and it is applicable to all operations which can be applied to points drawn from the respective distributions. We refer to our approach as {\em kernel probabilistic programming}. We illustrate it on synthetic data, and show how it can be used for nonparametric structural equation models, with an application to causal inference

Semiparametric estimation of a panel data proportional hazards model with fixed effects

This paper considers a panel duration model that has a proportional hazards specification with fixed effects. The paper shows how to estimate the baseline and integrated baseline hazard functions without assuming that they belong to known, finitedimensional families of functions. Existing estimators assume that the baseline hazard function belongs to a known parametric family. Therefore, the estimators presented here are more general than existing ones. This paper also presents a method for estimating the parametric part of the proportional hazards model with dependent right censoring, under which the partial likelihood estimator is inconsistent. The paper presents some Monte Carlo evidence on the small sample performance of the new estimators

Nonparametric and semiparametric estimation with discrete regressors

This paper presents and discusses procedures for estimating regression curves when regressors are discrete and applies them to semiparametric inference problems. We show that pointwise root-n-consistency and global consistency of regression curve estimates are achieved without employing any smoothing, even for discrete regressors with unbounded support. These results still hold when smoothers are used, under much weaker conditions than those required with continuous regressors. Such estimates are useful in semiparametric inference problems. We discuss in detail the partially linear regression model and shape-invariant modelling. We also provide some guidance on estimation in semiparametric models where continuous and discrete regressors are present. The paper also includes a Monte Carlo study

Kernel Bayes' rule

A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on representations of probabilities in reproducing kernel Hilbert spaces. Probabilities are uniquely characterized by the mean of the canonical map to the RKHS. The prior and conditional probabilities are expressed in terms of RKHS functions of an empirical sample: no explicit parametric model is needed for these quantities. The posterior is likewise an RKHS mean of a weighted sample. The estimator for the expectation of a function of the posterior is derived, and rates of consistency are shown. Some representative applications of the kernel Bayes' rule are presented, including Baysian computation without likelihood and filtering with a nonparametric state-space model.Comment: 27 pages, 5 figure