On information-optimal scripting of actions

Best paper award.Animals and humans encounter many tasks which permit ritualized behaviours, essentially fixed action sequences or “scripts”, similar to options known from Reinforcement Learning, but proceeding without intermediate decisions. While running a script, they proceed in an open-loop fashion. However even when these are already known, an agent needs to decide whether to perform a basic action or to trigger a script regarding the particular task. Here we study if including such scripts (i.e. behaviour rituals) is advantageous from the point of view of the relevant information required to take the decision to start such a script depending on the tasks. To achieve this, we modify the relevant information framework including sequences of basic actions to the possible actions

Information parsimony in collaborative interaction

We investigate the information processing cost associated with performing a collaborative dyadic task at a specific utility level. We build our approach on the Relevant Information formalism, which combines Shannon's Information Theory and Markov Decision Processes, for modelling a dyadic interaction scenario in which two agents with independent controllers move an object together with fully redundant control. Results show that increasing dyad's collaboration decreases the information intake and vice versa, antagonistic behavior puts a strain on the information bandwidth capacity. The key role of the particular embodiment of the environment in this trade-off is demonstrated in a series of simulations with informationally parsimonious optimal controllers.Peer reviewedFinal Published versio

Effects of Anticipation in Individually Motivated Behaviour on Control and Survival in a Multi-Agent Scenario with Resource Constraints

This is an open access article distributed under the Creative Commons Attribution License CC BY 3.0 which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Self-organization and survival are inextricably bound to an agent’s ability to control and anticipate its environment. Here we assess both skills when multiple agents compete for a scarce resource. Drawing on insights from psychology, microsociology and control theory, we examine how different assumptions about the behaviour of an agent’s peers in the anticipation process affect subjective control and survival strategies. To quantify control and drive behaviour, we use the recently developed information-theoretic quantity of empowerment with the principle of empowerment maximization. In two experiments involving extensive simulations, we show that agents develop risk-seeking, risk-averse and mixed strategies, which correspond to greedy, parsimonious and mixed behaviour. Although the principle of empowerment maximization is highly generic, the emerging strategies are consistent with what one would expect from rational individuals with dedicated utility models. Our results support empowerment maximization as a universal drive for guided self-organization in collective agent systemsPeer reviewedFinal Published versio

Kernelizing LSPE λ

We propose the use of kernel-based methods as underlying function approximator in the least-squares based policy evaluation framework of LSPE(λ) and LSTD(λ). In particular we present the ‘kernelization’ of model-free LSPE(λ). The ‘kernelization’ is computationally made possible by using the subset of regressors approximation, which approximates the kernel using a vastly reduced number of basis functions. The core of our proposed solution is an efficient recursive implementation with automatic supervised selection of the relevant basis functions. The LSPE method is well-suited for optimistic policy iteration and can thus be used in the context of online reinforcement learning. We use the high-dimensional Octopus benchmark to demonstrate this

Information-theoretic Sensorimotor Foundations of Fitts' Law

© 2019 ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published is accessible via https://doi.org/10.1145/3290607.3313053We propose a novel, biologically plausible cost/fitness function for sensorimotor control, formalized with the information-theoretic principle of empowerment, a task-independent universal utility. Empowerment captures uncertainty in the perception-action loop of different nature (e.g. noise, delays, etc.) in a single quantity. We present the formalism in a Fitts' law type goal-directed arm movement task and suggest that empowerment is one potential underlying determinant of movement trajectory planning in the presence of signal-dependent sensorimotor noise. Simulation results demonstrate the temporal relation of empowerment and various plausible control strategies for this specific task

Training Kohonen feature maps in different topologies: an analysis using genetic algorithms

Original paper can be found at: http://portal.acm.org/citation.cfmid=64551

An Informational Study of the Evolution of Codes in Different Population Structures

Best Student Paper Award. Attribution-NonCommercial-NoDerivs 3.0 United StatesWe consider the problem of the evolution of a code within a structured population of agents. The agents try to maximise their information about their environment by acquiring information from the outputs of other agents in the population. A naive use of information-theoretic methods would assume that every agent knows how to “interpret” the information offered by other agents. However, this assumes that one “knows” which other agents one observes, and thus which code they use. In our model, however, we wish to preclude that: it is not clear which other agents an agent is observing, and the resulting usable information is therefore influenced by the universality of the code used and by which agents an agent is “listening” to

Don't Believe Everything You Hear : Preserving Relevant Information by Discarding Social Information

Integrating information gained by observing others via Social Bayesian Learning can be beneficial for an agent’s performance, but can also enable population wide information cascades that perpetuate false beliefs through the agent population. We show how agents can influence the observation network by changing their probability of observing others, and demonstrate the existence of a population-wide equilibrium, where the advantages and disadvantages of the Social Bayesian update are balanced. We also use the formalism of relevant information to illustrate how negative information cascades are characterized by processing increasing amounts of non-relevant informatio