Probabilistic Guarantees for Safe Deep Reinforcement Learning

Deep reinforcement learning has been successfully applied to many control tasks, but the application of such agents in safety-critical scenarios has been limited due to safety concerns. Rigorous testing of these controllers is challenging, particularly when they operate in probabilistic environments due to, for example, hardware faults or noisy sensors. We propose MOSAIC, an algorithm for measuring the safety of deep reinforcement learning agents in stochastic settings. Our approach is based on the iterative construction of a formal abstraction of a controller's execution in an environment, and leverages probabilistic model checking of Markov decision processes to produce probabilistic guarantees on safe behaviour over a finite time horizon. It produces bounds on the probability of safe operation of the controller for different initial configurations and identifies regions where correct behaviour can be guaranteed. We implement and evaluate our approach on agents trained for several benchmark control problems

Holistic debugging - enabling instruction set simulation for software quality assurance

We present holistic debugging, a novel method for observing execution of complex and distributed software. It builds on an instruction set simulator, which provides reproducible experiments and non-intrusive probing of state in a distributed system. Instruction set simulators, however, only provide low-level information, so a holistic debugger contains a translation framework that maps this information to higher abstraction level observation tools, such as source code debuggers. We have created Nornir, a proof-of-concept holistic debugger, built on the simulator Simics. For each observed process in the simulated system, Nornir creates an abstraction translation stack, with virtual machine translators that map machine-level storage contents (e.g. physical memory, registers) provided by Simics, to application-level data (e.g. virtual memory contents) by parsing the data structures of operating systems and virtual machines. Nornir includes a modified version of the GNU debugger (GDB), which supports non-intrusive symbolic debugging of distributed applications. Nornir's main interface is a debugger shepherd, a programmable interface that controls multiple debuggers, and allows users to coherently inspect the entire state of heterogeneous, distributed applications. It provides a robust observation platform for construction of new observation tools

Software Reuse across Robotic Platforms: Limiting the effects of diversity

Robots have diverse capabilities and complex interactions with their environment. Software development for robotic platforms is time consuming due to the complex nature of the tasks to be performed. Such an environment demands sound software engineering practices to produce high quality software. However software engineering in the robotics domain fails to facilitate any significant level of software reuse or portability. This paper identifies the major issues limiting software reuse in the robotics domain. Lack of standardisation, diversity of robotic platforms, and the subtle effects of environmental interaction all contribute to this problem. It is then shown that software components, fuzzy logic, and related techniques can be used together to address this problem. While complete software reuse is not possible, it is demonstrated that significant levels of software reuse can be obtained. Without an acceptable level of reuse or portability, software engineering in the robotics domain will not be able to meet the demands of a rapidly developing field. The work presented in this paper demonstrates a method for supporting software reuse across robotic platforms and hence facilitating improved software engineering practices