Identification of Invariant Sensorimotor Structures as a Prerequisite for the Discovery of Objects

Perceiving the surrounding environment in terms of objects is useful for any general purpose intelligent agent. In this paper, we investigate a fundamental mechanism making object perception possible, namely the identification of spatio-temporally invariant structures in the sensorimotor experience of an agent. We take inspiration from the Sensorimotor Contingencies Theory to define a computational model of this mechanism through a sensorimotor, unsupervised and predictive approach. Our model is based on processing the unsupervised interaction of an artificial agent with its environment. We show how spatio-temporally invariant structures in the environment induce regularities in the sensorimotor experience of an agent, and how this agent, while building a predictive model of its sensorimotor experience, can capture them as densely connected subgraphs in a graph of sensory states connected by motor commands. Our approach is focused on elementary mechanisms, and is illustrated with a set of simple experiments in which an agent interacts with an environment. We show how the agent can build an internal model of moving but spatio-temporally invariant structures by performing a Spectral Clustering of the graph modeling its overall sensorimotor experiences. We systematically examine properties of the model, shedding light more globally on the specificities of the paradigm with respect to methods based on the supervised processing of collections of static images.Comment: 24 pages, 10 figures, published in Frontiers Robotics and A

Jacob: An Educational Agent in a Virtual Environment

The Jacob Project involves the construction of a virtual environment where an animated human-like agent called Jacob gives instruction to the user. The project focuses on three issues: the software engineering aspects of building a virtual reality system, the integration of natural language interaction and other interaction modalities, and the use of agent technology. Jacob has been given a task model and an instruction model in order to teach a particular task. The results of the project can be generalized so that the agent can be used to instruct other tasks in other virtual environments

Interaction of agents and environments

A new abstract model of interaction between agents and environments considered as objects of different types is introduced. Agents are represented by means of labelled transition systems considered up to bisimilarity. The equivalence of agents is characterised in terms of an algebra of behaviours which is a continuous algebra with approximation and two operations: nondeterministic choice and prefixing. Environments are introduced as agents supplied with an insertion function which takes the behaviour of an agent and the behaviour of an environment as arguments and returns the new behaviour of an environment. Arbitrary continuous functions can be used as insertion functions, and we use functions defined by means of rewriting logic as computable ones. The transformation of environment behaviours defined by the insertion function also defines a new type of agent equivalence--- insertion equivalence. Two behaviours are insertion equivalent if they define the same transformation of an environment. The properties of this equivalence are studied. Three main types of insertion functions are used to develop interesting applications: one-step insertion, head insertion, and look-ahead insertion functions

Extended neural metastability in an embodied model of sensorimotor coupling

The hypothesis that brain organization is based on mechanisms of metastable synchronization in neural assemblies has been popularized during the last decades of neuroscientific research. Nevertheless, the role of body and environment for understanding the functioning of metastable assemblies is frequently dismissed. The main goal of this paper is to investigate the contribution of sensorimotor coupling to neural and behavioral metastability using a minimal computational model of plastic neural ensembles embedded in a robotic agent in a behavioral preference task. Our hypothesis is that, under some conditions, the metastability of the system is not restricted to the brain but extends to the system composed by the interaction of brain, body and environment. We test this idea, comparing an agent in continuous interaction with its environment in a task demanding behavioral flexibility with an equivalent model from the point of view of “internalist neuroscience.” A statistical characterization of our model and tools from information theory allow us to show how (1) the bidirectional coupling between agent and environment brings the system closer to a regime of criticality and triggers the emergence of additional metastable states which are not found in the brain in isolation but extended to the whole system of sensorimotor interaction, (2) the synaptic plasticity of the agent is fundamental to sustain open structures in the neural controller of the agent flexibly engaging and disengaging different behavioral patterns that sustain sensorimotor metastable states, and (3) these extended metastable states emerge when the agent generates an asymmetrical circular loop of causal interaction with its environment, in which the agent responds to variability of the environment at fast timescales while acting over the environment at slow timescales, suggesting the constitution of the agent as an autonomous entity actively modulating its sensorimotor coupling with the world. We conclude with a reflection about how our results contribute in a more general way to current progress in neuroscientific research