Compositional Performance Modelling with the TIPPtool

Stochastic process algebras have been proposed as compositional specification formalisms for performance models. In this paper, we describe a tool which aims at realising all beneficial aspects of compositional performance modelling, the TIPPtool. It incorporates methods for compositional specification as well as solution, based on state-of-the-art techniques, and wrapped in a user-friendly graphical front end. Apart from highlighting the general benefits of the tool, we also discuss some lessons learned during development and application of the TIPPtool. A non-trivial model of a real life communication system serves as a case study to illustrate benefits and limitations

Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

Stochastic ordinary differential equations in applied and computational mathematics

Using concrete examples, we discuss the current and potential use of stochastic ordinary differential equations (SDEs) from the perspective of applied and computational mathematics. Assuming only a minimal background knowledge in probability and stochastic processes, we focus on aspects that distinguish SDEs from their deterministic counterparts. To illustrate a multiscale modelling framework, we explain how SDEs arise naturally as diffusion limits in the type of discrete-valued stochastic models used in chemical kinetics, population dynamics, and, most topically, systems biology. We outline some key issues in existence, uniqueness and stability that arise when SDEs are used as physical models, and point out possible pitfalls. We also discuss the use of numerical methods to simulate trajectories of an SDE and explain how both weak and strong convergence properties are relevant for highly-efficient multilevel Monte Carlo simulations. We flag up what we believe to be key topics for future research, focussing especially on nonlinear models, parameter estimation, model comparison and multiscale simulation

Patch-based Hybrid Modelling of Spatially Distributed Systems by Using Stochastic HYPE - ZebraNet as an Example

Individual-based hybrid modelling of spatially distributed systems is usually expensive. Here, we consider a hybrid system in which mobile agents spread over the space and interact with each other when in close proximity. An individual-based model for this system needs to capture the spatial attributes of every agent and monitor the interaction between each pair of them. As a result, the cost of simulating this model grows exponentially as the number of agents increases. For this reason, a patch-based model with more abstraction but better scalability is advantageous. In a patch-based model, instead of representing each agent separately, we model the agents in a patch as an aggregation. This property significantly enhances the scalability of the model. In this paper, we convert an individual-based model for a spatially distributed network system for wild-life monitoring, ZebraNet, to a patch-based stochastic HYPE model with accurate performance evaluation. We show the ease and expressiveness of stochastic HYPE for patch-based modelling of hybrid systems. Moreover, a mean-field analytical model is proposed as the fluid flow approximation of the stochastic HYPE model, which can be used to investigate the average behaviour of the modelled system over an infinite number of simulation runs of the stochastic HYPE model.Comment: In Proceedings QAPL 2014, arXiv:1406.156

Approximating evolutionary dynamics on networks using a Neighbourhood Configuration model

Evolutionary dynamics have been traditionally studied on homogeneously mixed and infinitely large populations. However, real populations are usually finite and characterised by complex interactions among individuals. Recent studies have shown that the outcome of the evolutionary process might be significantly affected by the population structure. Although an analytic investigation of the process is possible when the contact structure of the population has a simple form, this is usually infeasible on complex structures and the use of various assumptions and approximations is necessary. In this paper, we adopt an approximation method which has been recently used for the modelling of infectious disease transmission, to model evolutionary game dynamics on complex networks. Comparisons of the predictions of the model constructed with the results of computer simulations reveal the effectiveness of the process and the improved accuracy that it provides when, for example, compared to well-known pair approximation methods. This modelling framework offers a flexible way to carry out a systematic analysis of evolutionary game dynamics on graphs and to establish the link between network topology and potential system behaviours. As an example, we investigate how the Hawk and Dove strategies in a Hawk-Dove game spread in a population represented by a random regular graph, a random graph and a scale-free network, and we examine the features of the graph which affect the evolution of the population in this particular game