Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions

A multi-agent partially observable Markov decision process (MPOMDP) is a modeling paradigm used for high-level planning of heterogeneous autonomous agents subject to uncertainty and partial observation. Despite their modeling efficiency, MPOMDPs have not received significant attention in safety-critical settings. In this paper, we use barrier functions to design policies for MPOMDPs that ensure safety. Notably, our method does not rely on discretization of the belief space, or finite memory. To this end, we formulate sufficient and necessary conditions for the safety of a given set based on discrete-time barrier functions (DTBFs) and we demonstrate that our formulation also allows for Boolean compositions of DTBFs for representing more complicated safe sets. We show that the proposed method can be implemented online by a sequence of one-step greedy algorithms as a standalone safe controller or as a safety-filter given a nominal planning policy. We illustrate the efficiency of the proposed methodology based on DTBFs using a high-fidelity simulation of heterogeneous robots.Comment: 8 pages and 4 figure

A Formal Approach to Verification and Validation of Guidance, Navigation, and Control Algorithms

The traditional Monte Carlo based approaches to Verification & Validation (V&V) of Guidance Navigation and Control (GN&C) algorithms suffers from drawbacks, including typically requiring a significant amount of computational resources to guarantee a candidate algorithm’s appropriateness. Formal approaches to V&V of GN&C algorithms can help address these is-sues as they are not based on simulation. Therefore, we are investigating and developing an innovative formal V&V algorithm for spacecraft GN&C, specifically in the determination of safety of maneuvers for satellite Remote Proximity Operations and Docking (RPOD). Formal V&V methods could provide rigorous and quantifiable assurances of safety for a given satellite maneuver without the need to perform extensive simulations, enhancing the autonomous decision-making capability of a spacecraft with limited computational resources. The research leverages a novel approach to the forward stochastic reachability analysis problem utilizing Fourier transforms. Initial results indicate quantifiable assurance of safety for a maneuvering satellite reach and reach-avoid problem can be achieved that match (sometimes conservatively) the Monte Carlo runs but use up to three or more orders of magnitude less computation resources

StocHy: automated verification and synthesis of stochastic processes

StocHy is a software tool for the quantitative analysis of discrete-time stochastic hybrid systems (SHS). StocHy accepts a high-level description of stochastic models and constructs an equivalent SHS model. The tool allows to (i) simulate the SHS evolution over a given time horizon; and to automatically construct formal abstractions of the SHS. Abstractions are then employed for (ii) formal verification or (iii) control (policy, strategy) synthesis. StocHy allows for modular modelling, and has separate simulation, verification and synthesis engines, which are implemented as independent libraries. This allows for libraries to be easily used and for extensions to be easily built. The tool is implemented in C++ and employs manipulations based on vector calculus, the use of sparse matrices, the symbolic construction of probabilistic kernels, and multi-threading. Experiments show StocHy's markedly improved performance when compared to existing abstraction-based approaches: in particular, StocHy beats state-of-the-art tools in terms of precision (abstraction error) and computational effort, and finally attains scalability to large-sized models (12 continuous dimensions). StocHy is available at www.gitlab.com/natchi92/StocHy