Tracking Information Flow through the Environment: Simple Cases of Stigmerg

Recent work in sensor evolution aims at studying the perception-action loop in a formalized information-theoretic manner. By treating sensors as extracting information and actuators as having the capability to "imprint" information on the environment we can view agents as creating, maintaining and making use of various information flows. In our paper we study the perception-action loop of agents using Shannon information flows. We use information theory to track and reveal the important relationships between agents and their environment. For example, we provide an information-theoretic characterization of stigmergy and evolve finite-state automata as agent controllers to engage in stigmergic communication. Our analysis of the evolved automata and the information flow provides insight into how evolution organizes sensoric information acquisition, implicit internal and external memory, processing and action selection

PALMER: Perception-Action Loop with Memory for Long-Horizon Planning

To achieve autonomy in a priori unknown real-world scenarios, agents should be able to: i) act from high-dimensional sensory observations (e.g., images), ii) learn from past experience to adapt and improve, and iii) be capable of long horizon planning. Classical planning algorithms (e.g. PRM, RRT) are proficient at handling long-horizon planning. Deep learning based methods in turn can provide the necessary representations to address the others, by modeling statistical contingencies between observations. In this direction, we introduce a general-purpose planning algorithm called PALMER that combines classical sampling-based planning algorithms with learning-based perceptual representations. For training these perceptual representations, we combine Q-learning with contrastive representation learning to create a latent space where the distance between the embeddings of two states captures how easily an optimal policy can traverse between them. For planning with these perceptual representations, we re-purpose classical sampling-based planning algorithms to retrieve previously observed trajectory segments from a replay buffer and restitch them into approximately optimal paths that connect any given pair of start and goal states. This creates a tight feedback loop between representation learning, memory, reinforcement learning, and sampling-based planning. The end result is an experiential framework for long-horizon planning that is significantly more robust and sample efficient compared to existing methods.Comment: Website: https://palmer.epfl.c

Information-theoretic Sensorimotor Foundations of Fitts' Law

© 2019 ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published is accessible via https://doi.org/10.1145/3290607.3313053We propose a novel, biologically plausible cost/fitness function for sensorimotor control, formalized with the information-theoretic principle of empowerment, a task-independent universal utility. Empowerment captures uncertainty in the perception-action loop of different nature (e.g. noise, delays, etc.) in a single quantity. We present the formalism in a Fitts' law type goal-directed arm movement task and suggest that empowerment is one potential underlying determinant of movement trajectory planning in the presence of signal-dependent sensorimotor noise. Simulation results demonstrate the temporal relation of empowerment and various plausible control strategies for this specific task

Auditory smiles trigger unconscious facial imitation

Smiles, produced by the bilateral contraction of the zygomatic major muscles, are one of the most powerful expressions of positive affect and affiliation and also one of the earliest to develop [1]. The perception-action loop responsible for the fast and spontaneous imitation of a smile is considered a core component of social cognition [2]. In humans, social interaction is overwhelmingly vocal, and the visual cues of a smiling face co-occur with audible articulatory changes on the speaking voice [3]. Yet remarkably little is known about how such 'auditory smiles' are processed and reacted to. We have developed a voice transformation technique that selectively simulates the spectral signature of phonation with stretched lips and report here how we have used this technique to study facial reactions to smiled and non-smiled spoken sentences, finding that listeners' zygomatic muscles tracked auditory smile gestures even when they did not consciously detect them

Formal Approaches to a Definition of Agents

This thesis is a contribution to the formalisation of the notion of an agent within the class of finite multivariate Markov chains. In accordance with the literature agents are are seen as entities that act, perceive, and are goaldirected. We present a new measure that can be used to identify entities (called i-entities). The intuition behind this is that entities are spatiotemporal patterns for which every part makes every other part more probable. The measure, complete local integration (CLI), is formally investigated within the more general setting of Bayesian networks. It is based on the specific local integration (SLI) which is measured with respect to a partition. CLI is the minimum value of SLI over all partitions. Upper bounds are constructively proven and a possible lower bound is proposed. We also prove a theorem that shows that completely locally integrated spatiotemporal patterns occur as blocks in specific partitions of the global trajectory. Conversely we can identify partitions of global trajectories for which every block is completely locally integrated. These global partitions are the finest partitions that achieve a SLI less or equal to their own SLI. We also establish the transformation behaviour of SLI under permutations of the nodes in the Bayesian network. We then go on to present three conditions on general definitions of entities. These are most prominently not fulfilled by sets of random variables i.e. the perception-action loop, which is often used to model agents, is too restrictive a setting. We instead propose that any general entity definition should in effect specify a subset of the set of all spatiotemporal patterns of a given multivariate Markov chain. Any such definition will then define what we call an entity set. The set of all completely locally integrated spatiotemporal patterns is one example of such a set. Importantly the perception-action loop also naturally induces such an entity set. We then propose formal definitions of actions and perceptions for arbitrary entity sets. We show that these are generalisations of notions defined for the perception-action loop by plugging the entity-set of the perception-action loop into our definitions. We also clearly state the properties that general entity-sets have but the perception-action loop entity set does not. This elucidates in what way we are generalising the perception-action loop. Finally we look at some very simple examples of bivariate Markov chains. We present the disintegration hierarchy, explain it via symmetries, and calculate the i-entities. Then we apply our definitions of perception and action to these i-entities

Informational drives for sensor evolution

© 2012 Massachusetts Institute of Technology Published under a Creative Commons Attribution 4.0 International (CC BY 4.0) licenseIt has been hypothesized that the evolution of sensors is a pivotal driver for the evolution of organisms, and especially, as a crucial part of the perception-action loop, a driver for cognitive development. The questions of why and how this is the case are important: what are the principles that push the evolution of sensorimotor systems? An interesting aspect of this problem is the co-option of sensors for functions other than those originally driving their development (e.g. the auditive sense of bats being employed as a 'visual' modality). Even more striking is the phenomenon found in nature of sensors being driven to the limits of precision, while starting from much simpler beginnings. While a large potential for diversification and exaptation is visible in the observed phenotypes, gaining a deeper understanding of why and how this can be achieved is a significant problem. In this present paper, we will introduce a formal and generic information-theoretic model for understanding potential drives of sensor evolution, both in terms of improving sensory ability and in terms of extending and/or shifting sensory function

What do You Want to do Today? : Relevant-Information Bookkeeping in Goal-Oriented Behaviour

We extend existing models and methods for the informational treatment of the perception-action loop to the case of goaloriented behaviour and introduce the notion of relevant goal information as the amount of information an agent necessarily has to maintain about its goal. Starting from the hypothesis that organisms use information economically, we study the structure of this information and how goal-information parsimony can guide behaviour. It is shown how these methods lead to a general definition and quantification of sub-goals and how the biologically motivated hypothesis of information parsimony gives rise to the emergence of behavioural properties such as least-commitment and goal-concealing

Effects of Anticipation in Individually Motivated Behaviour on Control and Survival in a Multi-Agent Scenario with Resource Constraints

This is an open access article distributed under the Creative Commons Attribution License CC BY 3.0 which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Self-organization and survival are inextricably bound to an agent’s ability to control and anticipate its environment. Here we assess both skills when multiple agents compete for a scarce resource. Drawing on insights from psychology, microsociology and control theory, we examine how different assumptions about the behaviour of an agent’s peers in the anticipation process affect subjective control and survival strategies. To quantify control and drive behaviour, we use the recently developed information-theoretic quantity of empowerment with the principle of empowerment maximization. In two experiments involving extensive simulations, we show that agents develop risk-seeking, risk-averse and mixed strategies, which correspond to greedy, parsimonious and mixed behaviour. Although the principle of empowerment maximization is highly generic, the emerging strategies are consistent with what one would expect from rational individuals with dedicated utility models. Our results support empowerment maximization as a universal drive for guided self-organization in collective agent systemsPeer reviewedFinal Published versio

A distributed stochastic perception-action loop model of cell motility

Abstract In this paper, we present a novel framework for the modeling of cell-migration, and more specifically the migration of human keratinocytes. The model decouples the embodiment of an artificial cell into two elements. A cell-body is implemented by two sets of springs forming a membrane and a supporting cortical-cytoskeleton, which allows for cell-body rigidity and flexibility. The leading-edge, a structure spreading around the cell-body, is simulated with a stochastic cellular-automata. It defines the migratory forces that pull the cell-body according to its local spread around the cell. The overall movement of the leading-edge depends on stochastic interaction with the environment and guides the whole cell movement through spatiotemporal integration of local forces. We demonstrate that our cell migration model allows for spontaneous symmetry-breaking and directed cell movement and has in-built obstacle-avoidance, closely mimicking the migration of living cells. The model is extended to simulate chemotactic behavior, the artificial cell can sense and move along a gradient with its trajectory depending on the cell shape, stiffness and leading-edge dynamics. In summary, we have developed a novel cell migration model with emergent properties, wherein local forces create an integrated cell movement. The presented interplay of the distributed physical and an informational embodiment is not limited in reach to the example of cell migration, but can of interest for design of perception-action loops and sensor evolution in general