Switching and diffusion models for gene regulation networks

We analyze a hierarchy of three regimes for modeling gene regulation. The most complete model is a continuous time, discrete state space, Markov jump process. An intermediate 'switch plus diffusion' model takes the form of a stochastic differential equation driven by an independent continuous time Markov switch. In the third 'switch plus ODE' model the switch remains but the diffusion is removed. The latter two models allow for multi-scale simulation where, for the sake of computational efficiency, system components are treated differently according to their abundance. The 'switch plus ODE' regime was proposed by Paszek (Modeling stochasticity in gene regulation: characterization in the terms of the underlying distribution function, Bulletin of Mathematical Biology, 2007), who analyzed the steady state behavior, showing that the mean was preserved but the variance only approximated that of the full model. Here, we show that the tools of stochastic calculus can be used to analyze first and second moments for all time. A technical issue to be addressed is that the state space for the discrete-valued switch is infinite. We show that the new 'switch plus diffusion' regime preserves the biologically relevant measures of mean and variance, whereas the 'switch plus ODE' model uniformly underestimates the variance in the protein level. We also show that, for biologically relevant parameters, the transient behaviour can differ significantly from the steady state, justifying our time-dependent analysis. Extra computational results are also given for a protein dimerization model that is beyond the scope of the current analysis

Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

The control of nonlinear dynamical systems remains a major challenge for autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies, which have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication and extremely over-parameterized models have come with the cost of an overall reduction in our ability to interpret the resulting policies. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems in order to break down complex dynamics into simpler components. We exploit the rich representational power of probabilistic graphical models and derive an expectation-maximization (EM) algorithm for learning a sequence model to capture the temporal structure of the data and automatically decompose nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro

HMM based scenario generation for an investment optimisation problem

This is the post-print version of the article. The official published version can be accessed from the link below - Copyright @ 2012 Springer-Verlag.The Geometric Brownian motion (GBM) is a standard method for modelling financial time series. An important criticism of this method is that the parameters of the GBM are assumed to be constants; due to this fact, important features of the time series, like extreme behaviour or volatility clustering cannot be captured. We propose an approach by which the parameters of the GBM are able to switch between regimes, more precisely they are governed by a hidden Markov chain. Thus, we model the financial time series via a hidden Markov model (HMM) with a GBM in each state. Using this approach, we generate scenarios for a financial portfolio optimisation problem in which the portfolio CVaR is minimised. Numerical results are presented.This study was funded by NET ACE at OptiRisk Systems

Improving market-based forecasts of short-term interest rates: time-varying stationarity and the predictive content of switching regime-expectations

Modeling short-term interest rates as following regime-switching processes has become increasingly popular. Theoretically, regime-switching models are able to capture rational expectations of infrequently occurring discrete events. Technically, they allow for potential time-varying stationarity. After discussing both aspects with reference to the recent literature, this paper provides estimations of various univariate regime-switching specifications for the German three-month money market rate and bivariate specifications additionally including the term spread. However, the main contribution is a multi-step out-of-sample forecasting competition. It turns out that forecasts are improved substantially when allowing for state-dependence. Particularly, the informational content of the term spread for future short rate changes can be exploited optimally within a multivariate regime-switching framework

Experiments on the Application of IOHMMs to Model Financial Returns Series

Input/Output Hidden Markov Models (IOHMMs) are conditional hidden Markov models in which the emission (and possibly the transition) probabilities can be conditioned on an input sequence. For example, these conditional distributions can be linear, logistic, or non-linear (using for example multi-layer neural networks). We compare the generalization performance of several models which are special cases of Input/Output Hidden Markov Models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden Markov model, and various IOHMMs. The experiments compare these models on predicting the conditional density of returns of market and sector indices. Note that the unconditional Gaussian estimates the first moment with the historical average. The results show that, although for the first moment the historical average gives the best results, for the higher moments, the IOHMMs yielded significantly better performance, as estimated by the out-of-sample likelihood. "Input/Output Hidden Markov Models" (IOHMMs) sont des modèles de Markov cachés pour lesquels les probabilités d'émission (et possiblement de transition) peuvent dépendre d'une séquence d'entrée. Par exemple, ces distributions conditionnelles peuvent être linéaires, logistique, ou non-linéaire (utilisant, par exemple, une réseau de neurones multi-couches). Nous comparons les performances de généralisation de plusieurs modèles qui sont des cas particuliers de IOHMMs pour des problèmes de prédictions de séries financières : une gaussienne inconditionnelle, une gaussienne linéaire conditionnelle, une mixture de gaussienne, une mixture de gaussiennes linéaires conditionnelles, un modèle de Markov caché, et divers IOHMMs. Les expériences comparent ces modèles sur leur prédictions de la densité conditionnelle des rendements des indices sectoriels et du marché. Notons qu'une gaussienne inconditionnelle estime le premier moment avec une moyenne historique. Les résultats montrent que, même si la moyenne historique donne les meilleurs résultats pour le premier moment, pour les moments d'ordres supérieurs les IOHMMs performent significativement mieux, comme estimé par la vraisemblance hors-échantillon.Input/Output Hidden Markov Model (IOHMM), financial series, volatility, Modèles de Markov cachés, IOHMM, séries financières, volatilité

Robot Introspection with Bayesian Nonparametric Vector Autoregressive Hidden Markov Models

Robot introspection, as opposed to anomaly detection typical in process monitoring, helps a robot understand what it is doing at all times. A robot should be able to identify its actions not only when failure or novelty occurs, but also as it executes any number of sub-tasks. As robots continue their quest of functioning in unstructured environments, it is imperative they understand what is it that they are actually doing to render them more robust. This work investigates the modeling ability of Bayesian nonparametric techniques on Markov Switching Process to learn complex dynamics typical in robot contact tasks. We study whether the Markov switching process, together with Bayesian priors can outperform the modeling ability of its counterparts: an HMM with Bayesian priors and without. The work was tested in a snap assembly task characterized by high elastic forces. The task consists of an insertion subtask with very complex dynamics. Our approach showed a stronger ability to generalize and was able to better model the subtask with complex dynamics in a computationally efficient way. The modeling technique is also used to learn a growing library of robot skills, one that when integrated with low-level control allows for robot online decision making.Comment: final version submitted to humanoids 201