On the Random Structure of Behavioural Transition Systems

Random graphs have the property that they are very predictable. Even by exploring a small part reliable observations are possible regarding their structure and size. An unfortunate observation is that standard models for random graphs, such as the  Erdos-Renyi model, do not reflect the structure of the graphs that describe distributed systems and protocols. In this paper we propose to use the parallel composition of such random graphs to model `real' state spaces. We show how we can use this structure to predict the size of state spaces, and we can use it to explain that software bugs are in practice far easier to find than predicted by the standard random graph models. By some practical experiments we show that our new random model is an improvement over the standard model in predicting properties of transition systems representing realistic systems

Adaptation to criticality through organizational invariance in embodied agents

Many biological and cognitive systems do not operate deep within one or other regime of activity. Instead, they are poised at critical points located at phase transitions in their parameter space. The pervasiveness of criticality suggests that there may be general principles inducing this behaviour, yet there is no well-founded theory for understanding how criticality is generated at a wide span of levels and contexts. In order to explore how criticality might emerge from general adaptive mechanisms, we propose a simple learning rule that maintains an internal organizational structure from a specific family of systems at criticality. We implement the mechanism in artificial embodied agents controlled by a neural network maintaining a correlation structure randomly sampled from an Ising model at critical temperature. Agents are evaluated in two classical reinforcement learning scenarios: the Mountain Car and the Acrobot double pendulum. In both cases the neural controller appears to reach a point of criticality, which coincides with a transition point between two regimes of the agent's behaviour. These results suggest that adaptation to criticality could be used as a general adaptive mechanism in some circumstances, providing an alternative explanation for the pervasive presence of criticality in biological and cognitive systems.Comment: arXiv admin note: substantial text overlap with arXiv:1704.0525

Fully flexible analysis of behavioural sequences based on parametric survival models with frailties—A tutorial

Recent automated systems allow collecting continuous data on individual animals with high accuracy over a long time. During this time, animals can be traced across different (discrete) types of behavioural states, with the duration in each state being known. Nevertheless, analyses of such sequences of states or behaviours may prove difficult. Classic Markov-chain methods have limitations in respect to incorporating “memory” (effects of past states), the duration in the states and accounting for dependencies. Dependencies occur in many data sets, where, for example a variety of individuals from different groups are observed and/or when an experiment is divided in different crossover treatment phases. So-called parametric survival analysis with frailties can incorporate aforementioned aspects in one coherent model. The time spent in a specific state (performing a specific behaviour) can be modelled in dependence of the subsequent state (transition probabilities) while incorporating how these transitions are influenced by experimental treatments. In addition, prior states can be used as predictor variables (accounting for past behaviour). Finally, random effects can be included to account for dependencies according to, for example individual identity, group/farm/laboratory or experimental period. Using interactions between random and fixed effects, the within- and between-subject variability of the transition probabilities can be estimated to indicate variation between and consistency within individual subjects (individuality and personality). Moreover, relative hazards describing transitions from one state to several potential follow-up states can be estimated. Behavioural sequences and their modulation by experimental situations can be studied accordingly. Using two exemplary data sets, the data type and structure adequate for parametric survival analysis are introduced and advice is given on how to specify and run such models. Overall, parametric survival analysis with frailties presents a modern and versatile approach that can revive sequential analysis. This will facilitate more detailed use of behavioural data and accordingly detect more subtle aspects of behaviour.Ministerie van Economische Zaken en Klimaat http://dx.doi.org/10.13039/501100016238Bundesamt für Lebensmittelsicherheit und Veterinärwesen http://dx.doi.org/10.13039/501100006454Peer Reviewe

Robot introspection through learned hidden Markov models

In this paper we describe a machine learning approach for acquiring a model of a robot behaviour from raw sensor data. We are interested in automating the acquisition of behavioural models to provide a robot with an introspective capability. We assume that the behaviour of a robot in achieving a task can be modelled as a finite stochastic state transition system. Beginning with data recorded by a robot in the execution of a task, we use unsupervised learning techniques to estimate a hidden Markov model (HMM) that can be used both for predicting and explaining the behaviour of the robot in subsequent executions of the task. We demonstrate that it is feasible to automate the entire process of learning a high quality HMM from the data recorded by the robot during execution of its task.The learned HMM can be used both for monitoring and controlling the behaviour of the robot. The ultimate purpose of our work is to learn models for the full set of tasks associated with a given problem domain, and to integrate these models with a generative task planner. We want to show that these models can be used successfully in controlling the execution of a plan. However, this paper does not develop the planning and control aspects of our work, focussing instead on the learning methodology and the evaluation of a learned model. The essential property of the models we seek to construct is that the most probable trajectory through a model, given the observations made by the robot, accurately diagnoses, or explains, the behaviour that the robot actually performed when making these observations. In the work reported here we consider a navigation task. We explain the learning process, the experimental setup and the structure of the resulting learned behavioural models. We then evaluate the extent to which explanations proposed by the learned models accord with a human observer's interpretation of the behaviour exhibited by the robot in its execution of the task

Bisimulation, Logic and Reachability Analysis for Markovian Systems

In the recent years, there have been a large amount of investigations on safety verification of uncertain continuous systems. In engineering and applied mathematics, this verification is called stochastic reachability analysis, while in computer science this is called probabilistic model checking (PMC). In the context of this work, we consider the two terms interchangeable. It is worthy to note that PMC has been mostly considered for discrete systems. Therefore, there is an issue of improving the application of computer science techniques in the formal verification of continuous stochastic systems. We present a new probabilistic logic of model theoretic nature. The terms of this logic express reachability properties and the logic formulas express statistical properties of terms. Moreover, we show that this logic characterizes a bisimulation relation for continuous time continuous space Markov processes. For this logic we define a new semantics using state space symmetries. This is a recent concept that was successfully used in model checking. Using this semantics, we prove a full abstraction result. Furthermore, we prove a result that can be used in model checking, namely that the bisimulation preserves the probabilities of the reachable sets

Clustered marginalization of minorities during social transitions induced by co-evolution of behaviour and network structure

Large-scale transitions in societies are associated with both individual behavioural change and restructuring of the social network. These two factors have often been considered independently, yet recent advances in social network research challenge this view. Here we show that common features of societal marginalization and clustering emerge naturally during transitions in a co-evolutionary adaptive network model. This is achieved by explicitly considering the interplay between individual interaction and a dynamic network structure in behavioural selection. We exemplify this mechanism by simulating how smoking behaviour and the network structure get reconfigured by changing social norms. Our results are consistent with empirical findings: The prevalence of smoking was reduced, remaining smokers were preferentially connected among each other and formed increasingly marginalised clusters. We propose that self-amplifying feedbacks between individual behaviour and dynamic restructuring of the network are main drivers of the transition. This generative mechanism for co-evolution of individual behaviour and social network structure may apply to a wide range of examples beyond smoking.Comment: 16 pages, 5 figure

Petri nets for systems and synthetic biology

We give a description of a Petri net-based framework for modelling and analysing biochemical pathways, which uni¯es the qualita- tive, stochastic and continuous paradigms. Each perspective adds its con- tribution to the understanding of the system, thus the three approaches do not compete, but complement each other. We illustrate our approach by applying it to an extended model of the three stage cascade, which forms the core of the ERK signal transduction pathway. Consequently our focus is on transient behaviour analysis. We demonstrate how quali- tative descriptions are abstractions over stochastic or continuous descrip- tions, and show that the stochastic and continuous models approximate each other. Although our framework is based on Petri nets, it can be applied more widely to other formalisms which are used to model and analyse biochemical networks

Thermal performance of a naturally ventilated building using a combined algorithm of probabilistic occupant behaviour and deterministic heat and mass balance models

This study explores the role of occupant behaviour in relation to natural ventilation and its effects on summer thermal performance of naturally ventillated buildings. We develop a behavioural algorithm (the Yun algorithm) representing probablistic occupant behaviour and implement this within a dynamic energy simulation tool. A core of this algorithm is the use of Markov chain and Monte Carlo methods in order to integrate probablistic window use models into dynamic energy simulation procedures. The comparison between predicted and monitored window use patterns shows good agreement. Performance of the Yn algorithm is demonstrated for active, medium and passive window users and a range of office constructions. Results indicate, for example, that in some cases, the temperature of an office occupied by the active window user in summer is up to 2.6ºC lower than that for the passive window user. A comparison is made with results from an alernative bahavioural algorithm developed by Humphreys [H.B. Rijal, P. Tuohy, M.A. Humphreys, J.F. Nicol, A. Samual, J. Clarke, Using results from field surveys to predict the effect of open windows on thermal comfort and energy use in buildings, Energy and Buildings 39(7)(2007) 823-836.]. In general, the two algorithms lead to similar predictions, but the results suggest that the Yun algorithm better reflects the observed time of day effects on window use (i.e. the increased probability of action on arrival)

Are complex systems hard to evolve?

Evolutionary complexity is here measured by the number of trials/evaluations needed for evolving a logical gate in a non-linear medium. Behavioural complexity of the gates evolved is characterised in terms of cellular automata behaviour. We speculate that hierarchies of behavioural and evolutionary complexities are isomorphic up to some degree, subject to substrate specificity of evolution and the spectrum of evolution parameters