New insights on stochastic reachability

In this paper, we give new characterizations of the stochastic reachability problem for stochastic hybrid systems in the language of different theories that can be employed in studying stochastic processes (Markov processes, potential theory, optimal control). These characterizations are further used to obtain the probabilities involved in the context of stochastic reachability as viscosity solutions of some variational inequalities

Maximizing the probability of attaining a target prior to extinction

We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there exists a deterministic stationary policy that achieves the maximum value of this probability. We demonstrate how the maximization of this probability can be computed through the maximization of an expected total reward until the first hitting time to either the target or the cemetery set. Martingale characterizations of thrifty, equalizing, and optimal policies in the context of our problem are also established.Comment: 22 pages, 1 figure. Revise

Different Approaches on Stochastic Reachability as an Optimal Stopping Problem

Reachability analysis is the core of model checking of time systems. For stochastic hybrid systems, this safety verification method is very little supported mainly because of complexity and difficulty of the associated mathematical problems. In this paper, we develop two main directions of studying stochastic reachability as an optimal stopping problem. The first approach studies the hypotheses for the dynamic programming corresponding with the optimal stopping problem for stochastic hybrid systems. In the second approach, we investigate the reachability problem considering approximations of stochastic hybrid systems. The main difficulty arises when we have to prove the convergence of the value functions of the approximating processes to the value function of the initial process. An original proof is provided

Probabilistic Reachability Analysis for Large Scale Stochastic Hybrid Systems

This paper studies probabilistic reachability analysis for large scale stochastic hybrid systems (SHS) as a problem of rare event estimation. In literature, advanced rare event estimation theory has recently been embedded within a stochastic analysis framework, and this has led to significant novel results in rare event estimation for a diffusion process using sequential MC simulation. This paper presents this rare event estimation theory directly in terms of probabilistic reachability analysis of an SHS, and develops novel theory which allows to extend the novel results for application to a large scale SHS where a very huge number of rare discrete modes may contribute significantly to the reach probability. Essentially, the approach taken is to introduce an aggregation of the discrete modes, and to develop importance sampling relative to the rare switching between the aggregation modes. The practical working of this approach is demonstrated for the safety verification of an advanced air traffic control example

Verification of Stochastic Reach-Avoid Using RKHS Embeddings

A solution to the terminal-hitting and first-hitting stochastic reach-avoid problem for a Markov control process is presented. This solution takes advantage of a nonparametric representation of the stochastic kernel as a conditional distribution embedding within a reproducing kernel Hilbert space (RKHS). Because the disturbance is modeled as a data-driven stochastic process, this representation avoids intractable integrals in the dynamic recursion of the reach-avoid problem since the expectations can be calculated as an inner product within the RKHS. An example using a high-dimensional chain of integrators is presented, as well as for Clohessy-Wiltshire-Hill (CWH) dynamics