Hybrid Pathwise Sensitivity Methods for Discrete Stochastic Models of Chemical Reaction Systems

Stochastic models are often used to help understand the behavior of intracellular biochemical processes. The most common such models are continuous time Markov chains (CTMCs). Parametric sensitivities, which are derivatives of expectations of model output quantities with respect to model parameters, are useful in this setting for a variety of applications. In this paper, we introduce a class of hybrid pathwise differentiation methods for the numerical estimation of parametric sensitivities. The new hybrid methods combine elements from the three main classes of procedures for sensitivity estimation, and have a number of desirable qualities. First, the new methods are unbiased for a broad class of problems. Second, the methods are applicable to nearly any physically relevant biochemical CTMC model. Third, and as we demonstrate on several numerical examples, the new methods are quite efficient, particularly if one wishes to estimate the full gradient of parametric sensitivities. The methods are rather intuitive and utilize the multilevel Monte Carlo philosophy of splitting an expectation into separate parts and handling each in an efficient manner.Comment: 30 pages. The numerical example section has been extensively rewritte

A New Optimal Stepsize For Approximate Dynamic Programming

Approximate dynamic programming (ADP) has proven itself in a wide range of applications spanning large-scale transportation problems, health care, revenue management, and energy systems. The design of effective ADP algorithms has many dimensions, but one crucial factor is the stepsize rule used to update a value function approximation. Many operations research applications are computationally intensive, and it is important to obtain good results quickly. Furthermore, the most popular stepsize formulas use tunable parameters and can produce very poor results if tuned improperly. We derive a new stepsize rule that optimizes the prediction error in order to improve the short-term performance of an ADP algorithm. With only one, relatively insensitive tunable parameter, the new rule adapts to the level of noise in the problem and produces faster convergence in numerical experiments.Comment: Matlab files are included with the paper sourc

How close are time series to power tail L\'evy diffusions?

This article presents a new and easily implementable method to quantify the so-called coupling distance between the law of a time series and the law of a differential equation driven by Markovian additive jump noise with heavy-tailed jumps, such as $\alpha$-stable L\'evy flights. Coupling distances measure the proximity of the empirical law of the tails of the jump increments and a given power law distribution. In particular they yield an upper bound for the distance of the respective laws on path space. We prove rates of convergence comparable to the rates of the central limit theorem which are confirmed by numerical simulations. Our method applied to a paleoclimate time series of glacial climate variability confirms its heavy tail behavior. In addition this approach gives evidence for heavy tails in data sets of precipitable water vapor of the Western Tropical Pacific.Comment: 30 pages, 10 figure