59 research outputs found
State-Space Inference and Learning with Gaussian Processes
State-space inference and learning with Gaussian processes (GPs) is an unsolved problem. We propose a new, general methodology for inference and learning in nonlinear state-space models that are described probabilistically by non-parametric GP models. We apply the expectation maximization algorithm to iterate between inference in the latent state-space and learning the parameters of the underlying GP dynamics model. Copyright 2010 by the authors
Linear State-Space Model with Time-Varying Dynamics
This paper introduces a linear state-space model with time-varying dynamics.
The time dependency is obtained by forming the state dynamics matrix as a
time-varying linear combination of a set of matrices. The time dependency of
the weights in the linear combination is modelled by another linear Gaussian
dynamical model allowing the model to learn how the dynamics of the process
changes. Previous approaches have used switching models which have a small set
of possible state dynamics matrices and the model selects one of those matrices
at each time, thus jumping between them. Our model forms the dynamics as a
linear combination and the changes can be smooth and more continuous. The model
is motivated by physical processes which are described by linear partial
differential equations whose parameters vary in time. An example of such a
process could be a temperature field whose evolution is driven by a varying
wind direction. The posterior inference is performed using variational Bayesian
approximation. The experiments on stochastic advection-diffusion processes and
real-world weather processes show that the model with time-varying dynamics can
outperform previously introduced approaches.Comment: The final publication is available at Springer via
http://dx.doi.org/10.1007/978-3-662-44851-9_2
Predictive-State Decoders: Encoding the Future into Recurrent Networks
Recurrent neural networks (RNNs) are a vital modeling technique that rely on
internal states learned indirectly by optimization of a supervised,
unsupervised, or reinforcement training loss. RNNs are used to model dynamic
processes that are characterized by underlying latent states whose form is
often unknown, precluding its analytic representation inside an RNN. In the
Predictive-State Representation (PSR) literature, latent state processes are
modeled by an internal state representation that directly models the
distribution of future observations, and most recent work in this area has
relied on explicitly representing and targeting sufficient statistics of this
probability distribution. We seek to combine the advantages of RNNs and PSRs by
augmenting existing state-of-the-art recurrent neural networks with
Predictive-State Decoders (PSDs), which add supervision to the network's
internal state representation to target predicting future observations.
Predictive-State Decoders are simple to implement and easily incorporated into
existing training pipelines via additional loss regularization. We demonstrate
the effectiveness of PSDs with experimental results in three different domains:
probabilistic filtering, Imitation Learning, and Reinforcement Learning. In
each, our method improves statistical performance of state-of-the-art recurrent
baselines and does so with fewer iterations and less data.Comment: NIPS 201
- …