Goal-Directed Behavior under Variational Predictive Coding: Dynamic Organization of Visual Attention and Working Memory

Mental simulation is a critical cognitive function for goal-directed behavior because it is essential for assessing actions and their consequences. When a self-generated or externally specified goal is given, a sequence of actions that is most likely to attain that goal is selected among other candidates via mental simulation. Therefore, better mental simulation leads to better goal-directed action planning. However, developing a mental simulation model is challenging because it requires knowledge of self and the environment. The current paper studies how adequate goal-directed action plans of robots can be mentally generated by dynamically organizing top-down visual attention and visual working memory. For this purpose, we propose a neural network model based on variational Bayes predictive coding, where goal-directed action planning is formulated by Bayesian inference of latent intentional space. Our experimental results showed that cognitively meaningful competencies, such as autonomous top-down attention to the robot end effector (its hand) as well as dynamic organization of occlusion-free visual working memory, emerged. Furthermore, our analysis of comparative experiments indicated that introduction of visual working memory and the inference mechanism using variational Bayes predictive coding significantly improve the performance in planning adequate goal-directed actions

Anterior Intraparietal Area: a Hub in the Observed Manipulative Action Network.

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys\u2019 anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network

Editorial: Perceiving and Acting in the real world: from neural activity to behavior

The interaction between perception and action represents one of the pillars of human evolutionary success. Our interactions with the surrounding world involve a variety of behaviors, almost always including movements of the eyes and hands. Such actions rely on neural mechanisms that must process an enormous amount of information in order to generate appropriate motor commands. Yet, compared to the great advancements in the field of perception for cognition, the neural underpinnings of how we control our movements, as well as the interactions between perception and motor control, remain elusive. With this research topic we provide a framework for: 1) the perception of real objects and shapes using visual and haptic information, 2) the reference frames for action and perception, and 3) how perceived target properties are translated into goal-directed actions and object manipulation. The studies in this special issue employ a variety of methodologies that include behavioural kinematics, neuroimaging, transcranial magnetic stimulation and patient cases. Here we provide a brief summary and commentary on the articles included in this research topic

Interactions between motion and form processing in the human visual system

The predominant view of motion and form processing in the human visual system assumes that these two attributes are handled by separate and independent modules. Motion processing involves filtering by direction-selective sensors, followed by integration to solve the aperture problem. Form processing involves filtering by orientation-selective and size-selective receptive fields, followed by integration to encode object shape. It has long been known that motion signals can influence form processing in the well-known Gestalt principle of common fate; texture elements which share a common motion property are grouped into a single contour or texture region. However, recent research in psychophysics and neuroscience indicates that the influence of form signals on motion processing is more extensive than previously thought. First, the salience and apparent direction of moving lines depends on how the local orientation and direction of motion combine to match the receptive field properties of motion-selective neurons. Second, orientation signals generated by “motion-streaks” influence motion processing; motion sensitivity, apparent direction and adaptation are affected by simultaneously present orientation signals. Third, form signals generated by human body shape influence biological motion processing, as revealed by studies using point-light motion stimuli. Thus, form-motion integration seems to occur at several different levels of cortical processing, from V1 to STS

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesize that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of a ventral pathway dedicated to vision for perception, and a dorsal pathway dedicated to vision for action. To evaluate the functional organization in multi-task deep neural networks, we propose a method that measures the contribution of a unit towards each task, applying it to two networks that have been trained on either two related or two unrelated tasks, using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing. We conjecture that the method we propose can be used to analyze the anatomical and functional organization of the visual system and beyond. We predict that the degree to which tasks are related is a good descriptor of the degree to which they share downstream cortical-units.Comment: 16 pages, 5 figure

Parallel and convergent processing in grid cell, head-direction cell, boundary cell, and place cell networks.

The brain is able to construct internal representations that correspond to external spatial coordinates. Such brain maps of the external spatial topography may support a number of cognitive functions, including navigation and memory. The neuronal building block of brain maps are place cells, which are found throughout the hippocampus of rodents and, in a lower proportion, primates. Place cells typically fire in one or few restricted areas of space, and each area where a cell fires can range, along the dorsoventral axis of the hippocampus, from 30 cm to at least several meters. The sensory processing streams that give rise to hippocampal place cells are not fully understood, but substantial progress has been made in characterizing the entorhinal cortex, which is the gateway between neocortical areas and the hippocampus. Entorhinal neurons have diverse spatial firing characteristics, and the different entorhinal cell types converge in the hippocampus to give rise to a single, spatially modulated cell type-the place cell. We therefore suggest that parallel information processing in different classes of cells-as is typically observed at lower levels of sensory processing-continues up into higher level association cortices, including those that provide the inputs to hippocampus. WIREs Cogn Sci 2014, 5:207-219. doi: 10.1002/wcs.1272 Conflict of interest: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website

FMRI Reveals a Dissociation between Grasping and Perceiving the Size of Real 3D Objects

Background Almost 15 years after its formulation, evidence for the neuro-functional dissociation between a dorsal action stream and a ventral perception stream in the human cerebral cortex is still based largely on neuropsychological case studies. To date, there is no unequivocal evidence for separate visual computations of object features for performance of goal-directed actions versus perceptual tasks in the neurologically intact human brain. We used functional magnetic resonance imaging to test explicitly whether or not brain areas mediating size computation for grasping are distinct from those mediating size computation for perception. Methodology/Principal Findings Subjects were presented with the same real graspable 3D objects and were required to perform a number of different tasks: grasping, reaching, size discrimination, pattern discrimination or passive viewing. As in prior studies, the anterior intraparietal area (AIP) in the dorsal stream was more active during grasping, when object size was relevant for planning the grasp, than during reaching, when object properties were irrelevant for movement planning (grasping>reaching). Activity in AIP showed no modulation, however, when size was computed in the context of a purely perceptual task (size = pattern discrimination). Conversely, the lateral occipital (LO) cortex in the ventral stream was modulated when size was computed for perception (size>pattern discrimination) but not for action (grasping = reaching). Conclusions/Significance While areas in both the dorsal and ventral streams responded to the simple presentation of 3D objects (passive viewing), these areas were differentially activated depending on whether the task was grasping or perceptual discrimination, respectively. The demonstration of dual coding of an object for the purposes of action on the one hand and perception on the other in the same healthy brains offers a substantial contribution to the current debate about the nature of the neural coding that takes place in the dorsal and ventral streams