A predictive processing theory of sensorimotor contingencies: explaining the puzzle of perceptual presence and its absence in synesthesia

Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of “perceptual presence” has motivated “sensorimotor theories” which understand perception to involve the mastery of sensorimotor contingencies. However, the mechanistic basis of sensorimotor contingencies and their mastery has remained unclear. Sensorimotor theory also struggles to explain instances of perception, such as synesthesia, that appear to lack perceptual presence and for which relevant sensorimotor contingencies are difficult to identify. On alternative “predictive processing” theories, perceptual content emerges from probabilistic inference on the external causes of sensory signals, however, this view has addressed neither the problem of perceptual presence nor synesthesia. Here, I describe a theory of predictive perception of sensorimotor contingencies which (1) accounts for perceptual presence in normal perception, as well as its absence in synesthesia, and (2) operationalizes the notion of sensorimotor contingencies and their mastery. The core idea is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if those actions are not performed. These “counterfactually-rich” generative models encode sensorimotor contingencies related to repertoires of sensorimotor dependencies, with counterfactual richness determining the degree of perceptual presence associated with a stimulus. While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synesthetic concurrents are hypothesized to be counterfactually poor. In addition to accounting for the phenomenology of synesthesia, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. It may also lead to a new view of the (in)determinacy of normal perception

Baby steps: investigating the development of perceptual-motor couplings in infancy

There are cells in our motor cortex that fire both when we perform and when we observe similar actions. It has been suggested that these perceptual-motor couplings in the brain develop through associative learning during correlated sensorimotor experience. Although studies with adult participants have provided support for this hypothesis, there is no direct evidence that associative learning also underlies the initial formation of perceptual–motor couplings in the developing brain. With the present study we addressed this question by manipulating infants’ opportunities to associate the visual and motor representation of a novel action, and by investigating how this influenced their sensorimotor cortex activation when they observed this action performed by others. Pre-walking 7–9-month-old infants performed stepping movements on an infant treadmill while they either observed their own real-time leg movements (Contingent group) or the previously recorded leg movements of another infant (Non-contingent control group). Infants in a second control group did not perform any steps and only received visual experience with the stepping actions. Before and after the training period we measured infants’ sensorimotor alpha suppression, as an index of sensorimotor cortex activation, while they watched videos of other infants’ stepping actions. While we did not find greater sensorimotor alpha suppression following training in the Contingent group as a whole, we nevertheless found that the strength of the visuomotor contingency experienced during training predicted the amount of sensorimotor alpha suppression at post-test in this group. We did not find any effects of motor experience alone. These results suggest that the development of perceptual–motor couplings in the infant brain is likely to be supported by associative learning during correlated visuomotor experience

Grounding the Experience of a Visual Field through Sensorimotor Contingencies

Artificial perception is traditionally handled by hand-designing task specific algorithms. However, a truly autonomous robot should develop perceptive abilities on its own, by interacting with its environment, and adapting to new situations. The sensorimotor contingencies theory proposes to ground the development of those perceptive abilities in the way the agent can actively transform its sensory inputs. We propose a sensorimotor approach, inspired by this theory, in which the agent explores the world and discovers its properties by capturing the sensorimotor regularities they induce. This work presents an application of this approach to the discovery of a so-called visual field as the set of regularities that a visual sensor imposes on a naive agent's experience. A formalism is proposed to describe how those regularities can be captured in a sensorimotor predictive model. Finally, the approach is evaluated on a simulated system coarsely inspired from the human retina.Comment: 23 pages, 7 figures, published in Neurocomputin

An Electrocorticographic Brain Interface in an Individual with Tetraplegia

Brain-computer interface (BCI) technology aims to help individuals with disability to control assistive devices and reanimate paralyzed limbs. Our study investigated the feasibility of an electrocorticography (ECoG)-based BCI system in an individual with tetraplegia caused by C4 level spinal cord injury. ECoG signals were recorded with a high-density 32-electrode grid over the hand and arm area of the left sensorimotor cortex. The participant was able to voluntarily activate his sensorimotor cortex using attempted movements, with distinct cortical activity patterns for different segments of the upper limb. Using only brain activity, the participant achieved robust control of 3D cursor movement. The ECoG grid was explanted 28 days post-implantation with no adverse effect. This study demonstrates that ECoG signals recorded from the sensorimotor cortex can be used for real-time device control in paralyzed individuals

Sensorimotor Laws, Mechanisms, and Representations

According to the sensorimotor account, vision does not imply theconstruction of internally generated representations of the environment, butit isthe skillful exercise of the sensorimotor contingencies obeying sense-specific laws. In this short study, I focus on the notion of “sensorimotor law” and characterize the kind of explanation providedby the sensorimotor theory as a form of covering law model. I then question the nature of such sensorimotor laws and describe them as mechanisms. I show that a mechanistic interpretation provides a better account of the sensorimotor invariances, which fosters us to rebalance the explanatory burden of sensorimotor action and information. Finally, I show that the question of the roleof representations within the sensorimotor theory should be reconsidered

Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction

Despite its omnipresence in robotics application, the nature of spatial knowledgeand the mechanisms that underlie its emergence in autonomous agents are stillpoorly understood. Recent theoretical works suggest that the Euclidean structure ofspace induces invariants in an agent’s raw sensorimotor experience. We hypothesizethat capturing these invariants is beneficial for sensorimotor prediction and that,under certain exploratory conditions, a motor representation capturing the structureof the external space should emerge as a byproduct of learning to predict futuresensory experiences. We propose a simple sensorimotor predictive scheme, applyit to different agents and types of exploration, and evaluate the pertinence of thesehypotheses. We show that a naive agent can capture the topology and metricregularity of its sensor’s position in an egocentric spatial frame without any a prioriknowledge, nor extraneous supervision

Toward a self-organizing pre-symbolic neural model representing sensorimotor primitives

Copyright ©2014 Zhong, Cangelosi and Wermter.This is an open-access article distributed under the terms of the Creative Commons Attribution License (CCBY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these termsThe acquisition of symbolic and linguistic representations of sensorimotor behavior is a cognitive process performed by an agent when it is executing and/or observing own and others' actions. According to Piaget's theory of cognitive development, these representations develop during the sensorimotor stage and the pre-operational stage. We propose a model that relates the conceptualization of the higher-level information from visual stimuli to the development of ventral/dorsal visual streams. This model employs neural network architecture incorporating a predictive sensory module based on an RNNPB (Recurrent Neural Network with Parametric Biases) and a horizontal product model. We exemplify this model through a robot passively observing an object to learn its features and movements. During the learning process of observing sensorimotor primitives, i.e., observing a set of trajectories of arm movements and its oriented object features, the pre-symbolic representation is self-organized in the parametric units. These representational units act as bifurcation parameters, guiding the robot to recognize and predict various learned sensorimotor primitives. The pre-symbolic representation also accounts for the learning of sensorimotor primitives in a latent learning context.Peer reviewedFinal Published versio