Stochastic hybrid system : modelling and verification

Hybrid systems now form a classical computational paradigm unifying discrete and continuous system aspects. The modelling, analysis and verification of these systems are very difficult. One way to reduce the complexity of hybrid system models is to consider randomization. The need for stochastic models has actually multiple motivations. Usually, when building models complete information is not available and we have to consider stochastic versions. Moreover, non-determinism and uncertainty are inherent to complex systems. The stochastic approach can be thought of as a way of quantifying non-determinism (by assigning a probability to each possible execution branch) and managing uncertainty. This is built upon to the - now classical - approach in algorithmics that provides polynomial complexity algorithms via randomization. In this thesis we investigate the stochastic hybrid systems, focused on modelling and analysis. We propose a powerful unifying paradigm that combines analytical and formal methods. Its applications vary from air traffic control to communication networks and healthcare systems. The stochastic hybrid system paradigm has an explosive development. This is because of its very powerful expressivity and the great variety of possible applications. Each hybrid system model can be randomized in different ways, giving rise to many classes of stochastic hybrid systems. Moreover, randomization can change profoundly the mathematical properties of discrete and continuous aspects and also can influence their interaction. Beyond the profound foundational and semantics issues, there is the possibility to combine and cross-fertilize techniques from analytic mathematics (like optimization, control, adaptivity, stability, existence and uniqueness of trajectories, sensitivity analysis) and formal methods (like bisimulation, specification, reachability analysis, model checking). These constitute the major motivations of our research. We investigate new models of stochastic hybrid systems and their associated problems. The main difference from the existing approaches is that we do not follow one way (based only on continuous or discrete mathematics), but their cross-fertilization. For stochastic hybrid systems we introduce concepts that have been defined only for discrete transition systems. Then, techniques that have been used in discrete automata now come in a new analytical fashion. This is partly explained by the fact that popular verification methods (like theorem proving) can hardly work even on probabilistic extensions of discrete systems. When the continuous dimension is added, the idea to use continuous mathematics methods for verification purposes comes in a natural way. The concrete contribution of this thesis has four major milestones: 1. A new and a very general model for stochastic hybrid systems; 2. Stochastic reachability for stochastic hybrid systems is introduced together with an approximating method to compute reach set probabilities; 3. Bisimulation for stochastic hybrid systems is introduced and relationship with reachability analysis is investigated. 4. Considering the communication issue, we extend the modelling paradigm

Computational Techniques for Stochastic Reachability

As automated control systems grow in prevalence and complexity, there is an increasing demand for verification and controller synthesis methods to ensure these systems perform safely and to desired specifications. In addition, uncertain or stochastic behaviors are often exhibited (such as wind affecting the motion of an aircraft), making probabilistic verification desirable. Stochastic reachability analysis provides a formal means of generating the set of initial states that meets a given objective (such as safety or reachability) with a desired level of probability, known as the reachable (or safe) set, depending on the objective. However, the applicability of reachability analysis is limited in the scope and size of system it can address. First, generating stochastic reachable or viable sets is computationally intensive, and most existing methods rely on an optimal control formulation that requires solving a dynamic program, and which scales exponentially in the dimension of the state space. Second, almost no results exist for extending stochastic reachability analysis to systems with incomplete information, such that the controller does not have access to the full state of the system. This thesis addresses both of the above limitations, and introduces novel computational methods for generating stochastic reachable sets for both perfectly and partially observable systems. We initially consider a linear system with additive Gaussian noise, and introduce two methods for computing stochastic reachable sets that do not require dynamic programming. The first method uses a particle approximation to formulate a deterministic mixed integer linear program that produces an estimate to reachability probabilities. The second method uses a convex chance-constrained optimization problem to generate an under-approximation to the reachable set. Using these methods we are able to generate stochastic reachable sets for a four-dimensional spacecraft docking example in far less time than it would take had we used a dynamic program. We then focus on discrete time stochastic hybrid systems, which provide a flexible modeling framework for systems that exhibit mode-dependent behavior, and whose state space has both discrete and continuous components. We incorporate a stochastic observation process into the hybrid system model, and derive both theoretical and computational results for generating stochastic reachable sets subject to an observation process. The derivation of an information state allows us to recast the problem as one of perfect information, and we prove that solving a dynamic program over the information state is equivalent to solving the original problem. We then demonstrate that the dynamic program to solve the reachability problem for a partially observable stochastic hybrid system shares the same properties as for a partially observable Markov decision process (POMDP) with an additive cost function, and so we can exploit approximation strategies designed for POMDPs to solve the reachability problem. To do so, however, we first generate approximate representations of the information state and value function as either vectors or Gaussian mixtures, through a finite state approximation to the hybrid system or using a Gaussian mixture approximation to an indicator function defined over a convex region. For a system with linear dynamics and Gaussian measurement noise, we show that it exhibits special properties that do not require an approximation of the information state, which enables much more efficient computation of the reachable set. In all cases we provide convergence results and numerical examples

Different Approaches on Stochastic Reachability as an Optimal Stopping Problem

Reachability analysis is the core of model checking of time systems. For stochastic hybrid systems, this safety verification method is very little supported mainly because of complexity and difficulty of the associated mathematical problems. In this paper, we develop two main directions of studying stochastic reachability as an optimal stopping problem. The first approach studies the hypotheses for the dynamic programming corresponding with the optimal stopping problem for stochastic hybrid systems. In the second approach, we investigate the reachability problem considering approximations of stochastic hybrid systems. The main difficulty arises when we have to prove the convergence of the value functions of the approximating processes to the value function of the initial process. An original proof is provided

New insights on stochastic reachability

In this paper, we give new characterizations of the stochastic reachability problem for stochastic hybrid systems in the language of different theories that can be employed in studying stochastic processes (Markov processes, potential theory, optimal control). These characterizations are further used to obtain the probabilities involved in the context of stochastic reachability as viscosity solutions of some variational inequalities

Forward Stochastic Reachability Analysis for Uncontrolled Linear Systems using Fourier Transforms

We propose a scalable method for forward stochastic reachability analysis for uncontrolled linear systems with affine disturbance. Our method uses Fourier transforms to efficiently compute the forward stochastic reach probability measure (density) and the forward stochastic reach set. This method is applicable to systems with bounded or unbounded disturbance sets. We also examine the convexity properties of the forward stochastic reach set and its probability density. Motivated by the problem of a robot attempting to capture a stochastically moving, non-adversarial target, we demonstrate our method on two simple examples. Where traditional approaches provide approximations, our method provides exact analytical expressions for the densities and probability of capture.Comment: V3: HSCC 2017 (camera-ready copy), DOI updated, minor changes | V2: Review comments included | V1: 10 pages, 12 figure

A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates

This paper presents a methodology for safety verification of continuous and hybrid systems in the worst-case and stochastic settings. In the worst-case setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do not enter an unsafe region. No explicit computation of reachable sets is required in the construction of barrier certificates, which makes it possible to handle nonlinearity, uncertainty, and constraints directly within this framework. In the stochastic setting, our method computes an upper bound on the probability that a trajectory of the system reaches the unsafe set, a bound whose validity is proven by the existence of a barrier certificate. For polynomial systems, barrier certificates can be constructed using convex optimization, and hence the method is computationally tractable. Some examples are provided to illustrate the use of the method