42 research outputs found
Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations
We describe a method to perform functional operations on probability
distributions of random variables. The method uses reproducing kernel Hilbert
space representations of probability distributions, and it is applicable to all
operations which can be applied to points drawn from the respective
distributions. We refer to our approach as {\em kernel probabilistic
programming}. We illustrate it on synthetic data, and show how it can be used
for nonparametric structural equation models, with an application to causal
inference
Spectral learning of transducers over continuous sequences
In this paper we present a spectral algorithm for learning weighted nite state transducers (WFSTs) over paired input-output sequences, where the input is continuous and the output discrete. WFSTs are an important tool for modeling paired input-output sequences and have numerous applications in
real-world problems. Recently, Balle et al (2011) proposed a spectral method for learning WFSTs that overcomes some of the well known limitations of gradient-based or EM optimizations which can be computationally expensive and su er from local optima issues. Their algorithm can model distributions where both inputs and outputs are sequences from a discrete alphabet.
However, many real world problems require modeling paired sequences where the inputs are not discrete but continuos sequences. Modelling continuous sequences with spectral methods has been studied in the context of HMMs (Song et al 2010), where a spectral algorithm for this case was derived. In this
paper we follow that line of work and propose a spectral learning algorithm
for modelling paired input-output sequences where the inputs are continuous and the outputs are discrete. Our approach is based on generalizing the class of weighted nite state transducers over discrete input-output sequences to a class where transitions are linear combinations of elementary transitions and the weights of this linear combinations are determined by dynamic features of the continuous input sequence.
At its core, the algorithm is simple and scalable to large data sets. We present experiments on a real task that validate the eff ectiveness of the proposed approach.Postprint (published version
Modelling transition dynamics in MDPs with RKHS embeddings
We propose a new, nonparametric approach to learning and representing
transition dynamics in Markov decision processes (MDPs), which can be combined
easily with dynamic programming methods for policy optimisation and value
estimation. This approach makes use of a recently developed representation of
conditional distributions as \emph{embeddings} in a reproducing kernel Hilbert
space (RKHS). Such representations bypass the need for estimating transition
probabilities or densities, and apply to any domain on which kernels can be
defined. This avoids the need to calculate intractable integrals, since
expectations are represented as RKHS inner products whose computation has
linear complexity in the number of points used to represent the embedding. We
provide guarantees for the proposed applications in MDPs: in the context of a
value iteration algorithm, we prove convergence to either the optimal policy,
or to the closest projection of the optimal policy in our model class (an
RKHS), under reasonable assumptions. In experiments, we investigate a learning
task in a typical classical control setting (the under-actuated pendulum), and
on a navigation problem where only images from a sensor are observed. For
policy optimisation we compare with least-squares policy iteration where a
Gaussian process is used for value function estimation. For value estimation we
also compare to the NPDP method. Our approach achieves better performance in
all experiments.Comment: ICML201