Stochastic Representations of Ion Channel Kinetics and Exact Stochastic Simulation of Neuronal Dynamics

In this paper we provide two representations for stochastic ion channel kinetics, and compare the performance of exact simulation with a commonly used numerical approximation strategy. The first representation we present is a random time change representation, popularized by Thomas Kurtz, with the second being analogous to a "Gillespie" representation. Exact stochastic algorithms are provided for the different representations, which are preferable to either (a) fixed time step or (b) piecewise constant propensity algorithms, which still appear in the literature. As examples, we provide versions of the exact algorithms for the Morris-Lecar conductance based model, and detail the error induced, both in a weak and a strong sense, by the use of approximate algorithms on this model. We include ready-to-use implementations of the random time change algorithm in both XPP and Matlab. Finally, through the consideration of parametric sensitivity analysis, we show how the representations presented here are useful in the development of further computational methods. The general representations and simulation strategies provided here are known in other parts of the sciences, but less so in the present setting.Comment: 39 pages, 6 figures, appendix with XPP and Matlab cod

A patch that imparts unconditional stability to certain explicit integrators for SDEs

This paper proposes a simple strategy to simulate stochastic differential equations (SDE) arising in constant temperature molecular dynamics. The main idea is to patch an explicit integrator with Metropolis accept or reject steps. The resulting `Metropolized integrator' preserves the SDE's equilibrium distribution and is pathwise accurate on finite time intervals. As a corollary the integrator can be used to estimate finite-time dynamical properties along an infinitely long solution. The paper explains how to implement the patch (even in the presence of multiple-time-stepsizes and holonomic constraints), how it scales with system size, and how much overhead it requires. We test the integrator on a Lennard-Jones cluster of particles and `dumbbells' at constant temperature.Comment: 29 pages, 5 figure

Regression Monte Carlo for Microgrid Management

We study an islanded microgrid system designed to supply a small village with the power produced by photovoltaic panels, wind turbines and a diesel generator. A battery storage system device is used to shift power from times of high renewable production to times of high demand. We introduce a methodology to solve microgrid management problem using different variants of Regression Monte Carlo algorithms and use numerical simulations to infer results about the optimal design of the grid.Comment: CEMRACS 2017 Summer project - proceedings

On causal extrapolation of sequences with applications to forecasting

The paper suggests a method of extrapolation of notion of one-sided semi-infinite sequences representing traces of two-sided band-limited sequences; this features ensure uniqueness of this extrapolation and possibility to use this for forecasting. This lead to a forecasting method for more general sequences without this feature based on minimization of the mean square error between the observed path and a predicable sequence. These procedure involves calculation of this predictable path; the procedure can be interpreted as causal smoothing. The corresponding smoothed sequences allow unique extrapolations to future times that can be interpreted as optimal forecasts.Comment: arXiv admin note: substantial text overlap with arXiv:1111.670

Modeling delay in genetic networks: From delay birth-death processes to delay stochastic differential equations

Delay is an important and ubiquitous aspect of many biochemical processes. For example, delay plays a central role in the dynamics of genetic regulatory networks as it stems from the sequential assembly of first mRNA and then protein. Genetic regulatory networks are therefore frequently modeled as stochastic birth-death processes with delay. Here we examine the relationship between delay birth-death processes and their appropriate approximating delay chemical Langevin equations. We prove that the distance between these two descriptions, as measured by expectations of functionals of the processes, converges to zero with increasing system size. Further, we prove that the delay birth-death process converges to the thermodynamic limit as system size tends to infinity. Our results hold for both fixed delay and distributed delay. Simulations demonstrate that the delay chemical Langevin approximation is accurate even at moderate system sizes. It captures dynamical features such as the spatial and temporal distributions of transition pathways in metastable systems, oscillatory behavior in negative feedback circuits, and cross-correlations between nodes in a network. Overall, these results provide a foundation for using delay stochastic differential equations to approximate the dynamics of birth-death processes with delay

Multilevel Monte Carlo methods for applications in finance

Since Giles introduced the multilevel Monte Carlo path simulation method [18], there has been rapid development of the technique for a variety of applications in computational finance. This paper surveys the progress so far, highlights the key features in achieving a high rate of multilevel variance convergence, and suggests directions for future research.Comment: arXiv admin note: text overlap with arXiv:1202.6283; and with arXiv:1106.4730 by other author

Least-squares methods for policy iteration

Approximate reinforcement learning deals with the essential problem of applying reinforcement learning in large and continuous state-action spaces, by using function approximators to represent the solution. This chapter reviews least-squares methods for policy iteration, an important class of algorithms for approximate reinforcement learning. We discuss three techniques for solving the core, policy evaluation component of policy iteration, called: least-squares temporal difference, least-squares policy evaluation, and Bellman residual minimization. We introduce these techniques starting from their general mathematical principles and detailing them down to fully specified algorithms. We pay attention to online variants of policy iteration, and provide a numerical example highlighting the behavior of representative offline and online methods. For the policy evaluation component as well as for the overall resulting approximate policy iteration, we provide guarantees on the performance obtained asymptotically, as the number of samples processed and iterations executed grows to infinity. We also provide finite-sample results, which apply when a finite number of samples and iterations are considered. Finally, we outline several extensions and improvements to the techniques and methods reviewed

Branching processes, the max-plus algebra and network calculus

Branching processes can describe the dynamics of various queueing systems, peer-to-peer systems, delay tolerant networks, etc. In this paper we study the basic stochastic recursion of multitype branching processes, but in two non-standard contexts. First, we consider this recursion in the max-plus algebra where branching corresponds to finding the maximal offspring of the current generation. Secondly, we consider network-calculus-type deterministic bounds as introduced by Cruz, which we extend to handle branching-type processes. The paper provides both qualitative and quantitative results and introduces various applications of (max-plus) branching processes in queueing theory