Verification of Uncertain POMDPs Using Barrier Certificates

We consider a class of partially observable Markov decision processes (POMDPs) with uncertain transition and/or observation probabilities. The uncertainty takes the form of probability intervals. Such uncertain POMDPs can be used, for example, to model autonomous agents with sensors with limited accuracy, or agents undergoing a sudden component failure, or structural damage [1]. Given an uncertain POMDP representation of the autonomous agent, our goal is to propose a method for checking whether the system will satisfy an optimal performance, while not violating a safety requirement (e.g. fuel level, velocity, and etc.). To this end, we cast the POMDP problem into a switched system scenario. We then take advantage of this switched system characterization and propose a method based on barrier certificates for optimality and/or safety verification. We then show that the verification task can be carried out computationally by sum-of-squares programming. We illustrate the efficacy of our method by applying it to a Mars rover exploration example.Comment: 8 pages, 4 figure

A modular hybrid simulation framework for complex manufacturing system design

For complex manufacturing systems, the current hybrid Agent-Based Modelling and Discrete Event Simulation (ABM–DES) frameworks are limited to component and system levels of representation and present a degree of static complexity to study optimal resource planning. To address these limitations, a modular hybrid simulation framework for complex manufacturing system design is presented. A manufacturing system with highly regulated and manual handling processes, composed of multiple repeating modules, is considered. In this framework, the concept of modular hybrid ABM–DES technique is introduced to demonstrate a novel simulation method using a dynamic system of parallel multi-agent discrete events. In this context, to create a modular model, the stochastic finite dynamical system is extended to allow the description of discrete event states inside the agent for manufacturing repeating modules (meso level). Moreover, dynamic complexity regarding uncertain processing time and resources is considered. This framework guides the user step-by-step through the system design and modular hybrid model. A real case study in the cell and gene therapy industry is conducted to test the validity of the framework. The simulation results are compared against the data from the studied case; excellent agreement with 1.038% error margin is found in terms of the company performance. The optimal resource planning and the uncertainty of the processing time for manufacturing phases (exo level), in the presence of dynamic complexity is calculated

Strategy Synthesis for Autonomous Agents Using PRISM

We present probabilistic models for autonomous agent search and retrieve missions derived from Simulink models for an Unmanned Aerial Vehicle (UAV) and show how probabilistic model checking and the probabilistic model checker PRISM can be used for optimal controller generation. We introduce a sequence of scenarios relevant to UAVs and other autonomous agents such as underwater and ground vehicles. For each scenario we demonstrate how it can be modelled using the PRISM language, give model checking statistics and present the synthesised optimal controllers. We conclude with a discussion of the limitations when using probabilistic model checking and PRISM in this context and what steps can be taken to overcome them. In addition, we consider how the controllers can be returned to the UAV and adapted for use on larger search areas

Aximo: automated axiomatic reasoning for information update

Aximo is a software written in C++ that verifies epistemic properties of dynamic scenarios in multi-agent systems. The underlying logic of our tool is based on the algebraic axiomatics of Dynamic Epistemic Logic. We also present a new theoretical result: the worst case complexity of the verification problem of Aximo

Classical Knowledge for Quantum Security

We propose a decision procedure for analysing security of quantum cryptographic protocols, combining a classical algebraic rewrite system for knowledge with an operational semantics for quantum distributed computing. As a test case, we use our procedure to reason about security properties of a recently developed quantum secret sharing protocol that uses graph states. We analyze three different scenarios based on the safety assumptions of the classical and quantum channels and discover the path of an attack in the presence of an adversary. The epistemic analysis that leads to this and similar types of attacks is purely based on our classical notion of knowledge.Comment: extended abstract, 13 page

Mobile agent path planning under uncertain environment using reinforcement learning and probabilistic model checking

The major challenge in mobile agent path planning, within an uncertain environment, is effectively determining an optimal control model to discover the target location as quickly as possible and evaluating the control system's reliability. To address this challenge, we introduce a learning-verification integrated mobile agent path planning method to achieve both the effectiveness and the reliability. More specifically, we first propose a modified Q-learning algorithm (a popular reinforcement learning algorithm), called Q EA−learning algorithm, to find the best Q-table in the environment. We then determine the location transition probability matrix, and establish a probability model using the assumption that the agent selects a location with a higher Q-value. Secondly, the learnt behaviour of the mobile agent based on Q EA−learning algorithm, is formalized as a Discrete-time Markov Chain (DTMC) model. Thirdly, the required reliability requirements of the mobile agent control system are specified using Probabilistic Computation Tree Logic (PCTL). In addition, the DTMC model and the specified properties are taken as the input of the Probabilistic Model Checker PRISM for automatic verification. This is preformed to evaluate and verify the control system's reliability. Finally, a case study of a mobile agent walking in a grids map is used to illustrate the proposed learning algorithm. Here we have a special focus on the modelling approach demonstrating how PRISM can be used to analyse and evaluate the reliability of the mobile agent control system learnt via the proposed algorithm. The results show that the path identified using the proposed integrated method yields the largest expected reward.</p