Pathwise Sensitivity Analysis in Transient Regimes

The instantaneous relative entropy (IRE) and the corresponding instanta- neous Fisher information matrix (IFIM) for transient stochastic processes are pre- sented in this paper. These novel tools for sensitivity analysis of stochastic models serve as an extension of the well known relative entropy rate (RER) and the corre- sponding Fisher information matrix (FIM) that apply to stationary processes. Three cases are studied here, discrete-time Markov chains, continuous-time Markov chains and stochastic differential equations. A biological reaction network is presented as a demonstration numerical example

Stochastic time-evolution, information geometry and the Cramer-Rao Bound

We investigate the connection between the time-evolution of averages of stochastic quantities and the Fisher information and its induced statistical length. As a consequence of the Cramer-Rao bound, we find that the rate of change of the average of any observable is bounded from above by its variance times the temporal Fisher information. As a consequence of this bound, we obtain a speed limit on the evolution of stochastic observables: Changing the average of an observable requires a minimum amount of time given by the change in the average squared, divided by the fluctuations of the observable times the thermodynamic cost of the transformation. In particular for relaxation dynamics, which do not depend on time explicitly, we show that the Fisher information is a monotonically decreasing function of time and that this minimal required time is determined by the initial preparation of the system. We further show that the monotonicity of the Fisher information can be used to detect hidden variables in the system and demonstrate our findings for simple examples of continuous and discrete random processes.Comment: 25 pages, 4 figure

Path mutual information for a class of biochemical reaction networks

Living cells encode and transmit information in the temporal dynamics of biochemical components. Gaining a detailed understanding of the input-output relationship in biological systems therefore requires quantitative measures that capture the interdependence between complete time trajectories of biochemical components. Mutual information provides such a measure but its calculation in the context of stochastic reaction networks is associated with mathematical challenges. Here we show how to estimate the mutual information between complete paths of two molecular species that interact with each other through biochemical reactions. We demonstrate our approach using three simple case studies.Comment: 6 pages, 2 figure

Estimation of Continuous Time Models in Economics: an Overview

The dynamics of economic behaviour is often developed in theory as a continuous time system. Rigorous estimation and testing of such systems, and the analysis of some aspects of their properties, is of particular importance in distinguishing between competing hypotheses and the resulting models. The consequences for the international economy during the past eighteen months of failures in the financial sector, and particularly the banking sector, make it essential that the dynamics of financial and commodity markets and of macro-economic policy are well understood. The nonlinearity of the economic system means that it’s properties are heavily dependent on it’s parameter values. The estimators discussed here are tools to provide those parameter estimates.

Variational characterization of free energy: theory and algorithms

The article surveys and extends variational formulations of the thermodynamic free energy and discusses their information-theoretic content from the perspective of mathematical statistics. We revisit the well-known Jarzynski equality for nonequilibrium free energy sampling within the framework of importance sampling and Girsanov change-of-measure transformations. The implications of the different variational formulations for designing efficient stochastic optimization and nonequilibrium simulation algorithms for computing free energies are discussed and illustrated

A posteriori error analysis and adaptive non-intrusive numerical schemes for systems of random conservation laws

In this article we consider one-dimensional random systems of hyperbolic conservation laws. We first establish existence and uniqueness of random entropy admissible solutions for initial value problems of conservation laws which involve random initial data and random flux functions. Based on these results we present an a posteriori error analysis for a numerical approximation of the random entropy admissible solution. For the stochastic discretization, we consider a non-intrusive approach, the Stochastic Collocation method. The spatio-temporal discretization relies on the Runge--Kutta Discontinuous Galerkin method. We derive the a posteriori estimator using continuous reconstructions of the discrete solution. Combined with the relative entropy stability framework this yields computable error bounds for the entire space-stochastic discretization error. The estimator admits a splitting into a stochastic and a deterministic (space-time) part, allowing for a novel residual-based space-stochastic adaptive mesh refinement algorithm. We conclude with various numerical examples investigating the scaling properties of the residuals and illustrating the efficiency of the proposed adaptive algorithm