Different Approaches on Stochastic Reachability as an Optimal Stopping Problem

Reachability analysis is the core of model checking of time systems. For stochastic hybrid systems, this safety verification method is very little supported mainly because of complexity and difficulty of the associated mathematical problems. In this paper, we develop two main directions of studying stochastic reachability as an optimal stopping problem. The first approach studies the hypotheses for the dynamic programming corresponding with the optimal stopping problem for stochastic hybrid systems. In the second approach, we investigate the reachability problem considering approximations of stochastic hybrid systems. The main difficulty arises when we have to prove the convergence of the value functions of the approximating processes to the value function of the initial process. An original proof is provided

Stochastic approximations of hybrid systems

This paper introduces a method for approximating the dynamics of deterministic hybrid systems. Within this setting, we shall consider jump conditions that are characterized by spatial guards. After defining proper penalty functions along these deterministic guards, corresponding probabilistic intensities are introduced and the deterministic dynamics are approximated by the stochastic evolution of a continuous-time Markov process. We would illustrate how the definition of the stochastic barriers can avoid ill-posed events such as "grazing", and show how the probabilistic guards can be helpful in addressing the problem of event detection. Furthermore, this method represents a very general technique for handling Zeno phenomena; it provides a universal way to regularize a hybrid system. Simulations would show that the stochastic approximation of a hybrid system is accurate, while being able to handle ''pathological cases". Finally, further generalizations of this approach are motivated and discussed

On the origins of approximations for stochastic chemical kinetics

This paper considers the derivation of approximations for stochastic chemical kinetics governed by the discrete master equation. Here, the concepts of (1) partitioning on the basis of fast and slow reactions as opposed to fast and slow species and (2) conditional probability densities are used to derive approximate, partitioned master equations, which are Markovian in nature, from the original master equation. Under different conditions dictated by relaxation time arguments, such approximations give rise to both the equilibrium and hybrid (deterministic or Langevin equations coupled with discrete stochastic simulation) approximations previously reported. In addition, the derivation points out several weaknesses in previous justifications of both the hybrid and equilibrium systems and demonstrates the connection between the original and approximate master equations. Two simple examples illustrate situations in which these two approximate methods are applicable and demonstrate the two methods' efficiencies

Coherent spin states and stochastic hybrid path integrals

Stochastic hybrid systems involve a coupling between a discrete Markov chain and a continuous stochastic process. If the latter evolves deterministically between jumps in the discrete state, then the system reduces to a piecewise deterministic Markov process. Well known examples include stochastic gene expression, voltage fluctuations in neurons, and motor-driven intracellular transport. In this paper we use coherent spin states to construct a new path integral representation of the probability density functional for stochastic hybrid systems, which holds outside the weak noise regime. We use the path integral to derive a system of Langevin equations in the semi-classical limit, which extends previous diffusion approximations based on a quasi-steady-state reduction. We then show how in the weak noise limit the path integral is equivalent to an alternative representation that was previously derived using Doi–Peliti operators. The action functional of the latter is related to a large deviation principle for stochastic hybrid systems

Approximation and inference methods for stochastic biochemical kinetics-a tutorial review

Stochastic fluctuations of molecule numbers are ubiquitous in biological systems. Important examples include gene expression and enzymatic processes in living cells. Such systems are typically modelled as chemical reaction networks whose dynamics are governed by the chemical master equation. Despite its simple structure, no analytic solutions to the chemical master equation are known for most systems. Moreover, stochastic simulations are computationally expensive, making systematic analysis and statistical inference a challenging task. Consequently, significant effort has been spent in recent decades on the development of efficient approximation and inference methods. This article gives an introduction to basic modelling concepts as well as an overview of state of the art methods. First, we motivate and introduce deterministic and stochastic methods for modelling chemical networks, and give an overview of simulation and exact solution methods. Next, we discuss several approximation methods, including the chemical Langevin equation, the system size expansion, moment closure approximations, time-scale separation approximations and hybrid methods. We discuss their various properties and review recent advances and remaining challenges for these methods. We present a comparison of several of these methods by means of a numerical case study and highlight some of their respective advantages and disadvantages. Finally, we discuss the problem of inference from experimental data in the Bayesian framework and review recent methods developed the literature. In summary, this review gives a self-contained introduction to modelling, approximations and inference methods for stochastic chemical kinetics

Hybrid deterministic stochastic systems with microscopic look-ahead dynamics

We study the impact of stochastic mechanisms on a coupled hybrid system consisting of a general advection-diffusion-reaction partial differential equation and a spatially distributed stochastic lattice noise model. The stochastic dynamics include both spin-flip and spin-exchange type interparticle interactions. Furthermore, we consider a new, asymmetric, single exclusion pro- cess, studied elsewhere in the context of traffic flow modeling, with an one-sided interaction potential which imposes advective trends on the stochastic dynamics. This look-ahead stochastic mechanism is responsible for rich nonlinear behavior in solutions. Our approach relies heavily on first deriving approximate differential mesoscopic equations. These approximations become exact either in the long range, Kac interaction partial differential equation case, or, given sufficient time separation con- ditions, between the partial differential equation and the stochastic model giving rise to a stochastic averaging partial differential equation. Although these approximations can in some cases be crude, they can still give a first indication, via linearized stability analysis, of the interesting regimes for the stochastic model. Motivated by this linearized stability analysis we choose particular regimes where interacting nonlinear stochastic waves are responsible for phenomena such as random switching, convective instability, and metastability, all driven by stochasticity. Numerical kinetic Monte Carlo simulations of the coarse grained hybrid system are implemented to assist in producing solutions and understanding their behavior

Analysis of DC microgrids as stochastic hybrid systems

A modeling framework for dc microgrids and distribution systems based on the dual active bridge (DAB) topology is presented. The purpose of this framework is to accurately characterize dynamic behavior of multi-converter systems as a function of exogenous load and source inputs. The base model is derived for deterministic inputs and then extended for the case of stochastic load behavior. At the core of the modeling framework is a large-signal DAB model that accurately describes the dynamics of both ac and dc state variables. This model addresses limitations of existing DAB converter models, which are not suitable for system-level analysis due to inaccuracy and poor upward scalability. The converter model acts as a fundamental building block in a general procedure for constructing models of multi-converter systems. System-level model construction is only possible due to structural properties of the converter model that mitigate prohibitive increases in size and complexity. To characterize the impact of randomness in practical loads, stochastic load descriptions are included in the deterministic dynamic model. The combined behavior of distributed loads is represented by a continuous-time stochastic process. Models that govern this load process are generated using a new modeling procedure, which builds incrementally from individual device-level representations. To merge the stochastic load process and deterministic dynamic models, the microgrid is modeled as a stochastic hybrid system. The stochastic hybrid model predicts the evolution of moments of dynamic state variables as a function of load model parameters. Moments of dynamic states provide useful approximations of typical system operating conditions over time. Applications of the deterministic models include system stability analysis and computationally efficient time-domain simulation. The stochastic hybrid models provide a framework for performance assessment and optimization --Abstract, page iv