Improving Performance of Feedback-Based Real-Time Networks using Model Checking and Reinforcement Learning

Traditionally, automatic control techniques arose due to need for automation in mechanical systems. These techniques rely on robust mathematical modelling of physical systems with the goal to drive their behaviour to desired set-points. Decades of research have successfully automated, optimized, and ensured safety of a wide variety of mechanical systems. Recent advancement in digital technology has made computers pervasive into every facet of life. As such, there have been many recent attempts to incorporate control techniques into digital technology. This thesis investigates the intersection and co-application of control theory and computer science to evaluate and improve performance of time-critical systems. The thesis applies two different research areas, namely, model checking and reinforcement learning to design and evaluate two unique real-time networks in conjunction with control technologies. The first is a camera surveillance system with the goal of constrained resource allocation to self-adaptive cameras. The second is a dual-delay real-time communication network with the goal of safe packet routing with minimal delays.The camera surveillance system consists of self-adaptive cameras and a centralized manager, in which the cameras capture a stream of images and transmit them to a central manager over a shared constrained communication channel. The event-based manager allocates fractions of the shared bandwidth to all cameras in the network. The thesis provides guarantees on the behaviour of the camera surveillance network through model checking. Disturbances that arise during image capture due to variations in capture scenes are modelled using probabilistic and non-deterministic Markov Decision Processes (MDPs). The different properties of the camera network such as the number of frame drops and bandwidth reallocations are evaluated through formal verification.The second part of the thesis explores packet routing for real-time networks constructed with nodes and directed edges. Each edge in the network consists of two different delays, a worst-case delay that captures high load characteristics, and a typical delay that captures the current network load. Each node in the network takes safe routing decisions by considering delays already encountered and the amount of remaining time. The thesis applies reinforcement learning to route packets through the network with minimal delays while ensuring the total path delay from source to destination does not exceed the pre-determined deadline of the packet. The reinforcement learning algorithm explores new edges to find optimal routing paths while ensuring safety through a simple pre-processing algorithm. The thesis shows that it is possible to apply powerful reinforcement learning techniques to time-critical systems with expert knowledge about the system

An automata-based automatic verification environment

With the continuing growth of computer systems including safety-critical computer control systems, the need for reliable tools to help construct, analyze, and verify such systems also continues to grow. The basic motivation of this work is to build such a formal verification environment for computer-based systems. An example of such a tool is the Design Oriented Verification and Evaluation (DOVE) created by Australian Defense Science and Technology Organization. One of the advantages of DOVE is that it combines ease of use provided by a graphical user interface for describing specifications in the form of extended state machines with the rigor of proving linear temporal logic properties in a robust theorem prover, Isabelle which was developed at Cambridge University, UK, and TU Munich, Germany. A different class of examples is that of model checkers, such as SPIN and SMV. In this work, we describe our technique to increase the utility of DOVE by extending it with the capability to build systems by specifying components. This added utility is demonstrated with a concrete example from a real project to study aspects of the control unit for an infusion pump being built at the Walter Reid Army Institute of Research. Secondly, we provide a formulation of linear temporal logic (LTL) in the theorem prover Isabelle. Next, we present a formalization of a variation of the algorithm for translating LTL into Büchi automata. The original translation algorithm is presented in Gerth et al and is the basis of model checkers such as SPIN. We also provide a formal proof of the termination and correctness of this algorithm. All definitions and proofs have been done fully formally within the generic theorem prover Isabelle, which guarantees the rigor of our work and the reliability of the results obtained. Finally, we introduce the automata theoretic framework for automatic verification as our future works

Identification and separation of DNA mixtures using peak area information (Updated version of Statistical Research Paper No. 25)

We introduce a new methodology, based upon probabilistic expert systems, for analysing forensic identification problems involving DNA mixture traces using quantitative peak area information. Peak area is modelled with conditional Gaussian distributions. The expert system can be used for ascertaining whether individuals, whose profiles have been measured, have contributed to the mixture, but also to predict DNA profiles of unknown contributors by separating the mixture into its individual components. The potential of our probabilistic methodology is illustrated on case data examples and compared with alternative approaches. The advantages are that identification and separation issues can be handled in a unified way within a single probabilistic model and the uncertainty associated with the analysis is quantified. Further work, required to bring the methodology to a point where it could be applied to the routine analysis of casework, is discussed

PRISM-games:Verification and Strategy Synthesis for Stochastic Multi-player Games with Multiple Objectives

PRISM-games is a tool for modelling, verification and strategy synthesis for stochastic multi-player games. These allow models to incorporate both probability, to represent uncertainty, unreliability or randomisation, and game-theoretic aspects, for systems where different entities have opposing objectives. Applications include autonomous transport, security protocols, energy management systems and many more. We provide a detailed overview of the PRISM-games tool, including its modelling and property specification formalisms, and its underlying architecture and implementation. In particular, we discuss some of its key features, which include multi-objective and compositional approaches to verification and strategy synthesis. We also discuss the scalability and efficiency of the tool and give an overview of some of the case studies to which it has been applied

RRS "Charles Darwin" Cruise 169, 17 Feb-19 Mar 2005. Hydrothermal exploration of the southern Mid-Atlantic Ridge

The principal objective of this cruise was to identify the first site or sites of high temperature hydrothermal venting anywhere on the southern Mid-Atlantic Ridge, to characterize their geological setting, preliminary chemical nature and to identify, where possible, the nature of any vent-endemic species that might inhabit such vents to investigate whether this ridge system might represent a new biogeographic province. Initially we used the TOBI deep-tow sidescan system equipped with a CTD system and optical backscatter sensors, together with Miniature Autonomous Plume Recorders (MAPRs) to identify two new sites in which diagnostic chemically- and particle-laden plumes indicated the presence of high-temperature hydrothermal venting. Subsequently, we used the ABE autonomous underwater vehicle to (1) locate the core of one of these hydrothermal plumes, (2) obtain a detailed map of the underlying seafloor and (3) photograph three discrete hydrothermal sites (2 black-smoker systems, 1 diffuse-flow) and their associated ecosystems. A series of CTD stations were occupied for water column investigations and a number of rock-coring and dredging stations were also undertaken to provide groundtruthing of sidescan sonar images of the Mid-Atlantic Ridge seafloor

Methodologies for Analyzing Equilibria in Wireless Games

Under certain assumptions in terms of information and models, equilibria correspond to possible stable outcomes in conflicting or cooperative scenarios where rational entities interact. For wireless engineers, it is of paramount importance to be able to predict and even ensure such states at which the network will effectively operate. In this article, we provide non-exhaustive methodologies for characterizing equilibria in wireless games in terms of existence, uniqueness, selection, and efficiency.Comment: To appear in IEEE Signal Processing Magazine, Sep. 200