Model reduction for a class of singularly perturbed stochastic differential equations

A class of singularly perturbed stochastic differential equations (SDE) with linear drift and nonlinear diffusion terms is considered. We prove that, on a finite time interval, the trajectories of the slow variables can be well approximated by those of a system with reduced dimension as the singular perturbation parameter becomes small. In particular, we show that when this parameter becomes small the first and second moments of the reduced system's variables closely approximate the first and second moments, respectively, of the slow variables of the singularly perturbed system. Chemical Langevin equations describing the stochastic dynamics of molecular systems with linear propensity functions including both fast and slow reactions fall within the class of SDEs considered here. We therefore illustrate the goodness of our approximation on a simulation example modeling a well known biomolecular system with fast and slow processes.United States. Air Force Office of Scientific Research (Grant FA9550-14-1-0060)National Institute of General Medical Sciences (U.S.) (Grant P50 GMO9879

Model reduction for a class of singularly perturbed stochastic differential equations : Fast variable approximation (Extended Version)

This is an extended version of a paper of the same title accepted to American Control Conference (ACC) 2016

Singularly perturbed forward-backward stochastic differential equations: application to the optimal control of bilinear systems

We study linear-quadratic stochastic optimal control problems with bilinear state dependence for which the underlying stochastic differential equation (SDE) consists of slow and fast degrees of freedom. We show that, in the same way in which the underlying dynamics can be well approximated by a reduced order effective dynamics in the time scale limit (using classical homogenziation results), the associated optimal expected cost converges in the time scale limit to an effective optimal cost. This entails that we can well approximate the stochastic optimal control for the whole system by the reduced order stochastic optimal control, which is clearly easier to solve because of lower dimensionality. The approach uses an equivalent formulation of the Hamilton-Jacobi-Bellman (HJB) equation, in terms of forward-backward SDEs (FBSDEs). We exploit the efficient solvability of FBSDEs via a least squares Monte Carlo algorithm and show its applicability by a suitable numerical example

Optimal control of multiscale systems using reduced-order models

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy "first reduce, then optimize"--rather than "first optimize, then reduce"--is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the "first reduce, then control" strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the "first reduce, then optmize" approach and discuss possible pitfalls

Reduction of Markov chains with two-time-scale state transitions

In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as $\lambda\rightarrow\infty$. We divide the state space of the Markov chain $X$ into a fast state space and a slow state space and define a reduced chain $Y$ on the slow state space. Our main result is that the distribution of the original chain $X$ will converge in total variation distance to that of the reduced chain $Y$ uniformly in time $t$ as $\lambda\rightarrow\infty$.Comment: 30 pages, 3 figures; Stochastics: An International Journal of Probability and Stochastic Processes, 201

Control of singularly perturbed hybrid stochastic systems

In this paper, we study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental “ergodicity” property for a class of “infinitesimal control systems” associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite- and infinite-discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e., are characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach