1,238 research outputs found
Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
We propose a recurrent neural network for a "model-free" simulation of a
dynamical system with unknown parameters without prior knowledge. The deep
learning model aims to jointly learn the nonlinear time marching operator and
the effects of the unknown parameters from a time series dataset. We assume
that the time series data set consists of an ensemble of trajectories for a
range of the parameters. The learning task is formulated as a statistical
inference problem by considering the unknown parameters as random variables. A
latent variable is introduced to model the effects of the unknown parameters,
and a variational inference method is employed to simultaneously train
probabilistic models for the time marching operator and an approximate
posterior distribution for the latent variable. Unlike the classical
variational inference, where a factorized distribution is used to approximate
the posterior, we employ a feedforward neural network supplemented by an
encoder recurrent neural network to develop a more flexible probabilistic
model. The approximate posterior distribution makes an inference on a
trajectory to identify the effects of the unknown parameters. The time marching
operator is approximated by a recurrent neural network, which takes a latent
state sampled from the approximate posterior distribution as one of the input
variables, to compute the time evolution of the probability distribution
conditioned on the latent variable. In the numerical experiments, it is shown
that the proposed variational inference model makes a more accurate simulation
compared to the standard recurrent neural networks. It is found that the
proposed deep learning model is capable of correctly identifying the dimensions
of the random parameters and learning a representation of complex time series
data
Recurrent Gaussian processes
We define Recurrent Gaussian Processes (RGP) models, a general family of
Bayesian nonparametric models with recurrent GP priors which are able to learn
dynamical patterns from sequential data. Similar to Recurrent Neural Networks
(RNNs), RGPs can have different formulations for their internal states,
distinct inference methods and be extended with deep structures. In such
context, we propose a novel deep RGP model whose autoregressive states are
latent, thereby performing representation and dynamical learning
simultaneously. To fully exploit the Bayesian nature of the RGP model we
develop the Recurrent Variational Bayes (REVARB) framework, which enables
efficient inference and strong regularization through coherent propagation of
uncertainty across the RGP layers and states. We also introduce a RGP extension
where variational parameters are greatly reduced by being reparametrized
through RNN-based sequential recognition models. We apply our model to the
tasks of nonlinear system identification and human motion modeling. The
promising obtained results indicate that our RGP model maintains its highly
flexibility while being able to avoid overfitting and being applicable even
when larger datasets are not available
Recurrent Gaussian Processes
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference methods and be extended with deep structures. In such context, we propose a novel deep RGP model whose autoregressive states are latent, thereby performing representation and dynamical learning simultaneously. To fully exploit the Bayesian nature of the RGP model we develop the Recurrent Variational Bayes (REVARB) framework, which enables efficient inference and strong regularization through coherent propagation of uncertainty across the RGP layers and states. We also introduce a RGP extension where variational parameters are greatly reduced by being reparametrized through RNN-based sequential recognition models. We apply our model to the tasks of nonlinear system identification and human motion modeling. The promising obtained results indicate that our RGP model maintains its highly flexibility while being able to avoid overfitting and being applicable even when larger datasets are not available
- …