Visual Affordance Prediction for Guiding Robot Exploration

Motivated by the intuitive understanding humans have about the space of possible interactions, and the ease with which they can generalize this understanding to previously unseen scenes, we develop an approach for learning visual affordances for guiding robot exploration. Given an input image of a scene, we infer a distribution over plausible future states that can be achieved via interactions with it. We use a Transformer-based model to learn a conditional distribution in the latent embedding space of a VQ-VAE and show that these models can be trained using large-scale and diverse passive data, and that the learned models exhibit compositional generalization to diverse objects beyond the training distribution. We show how the trained affordance model can be used for guiding exploration by acting as a goal-sampling distribution, during visual goal-conditioned policy learning in robotic manipulation.Comment: Old Paper; Presented in ICRA 202

Finetuning Offline World Models in the Real World

Reinforcement Learning (RL) is notoriously data-inefficient, which makes training on a real robot difficult. While model-based RL algorithms (world models) improve data-efficiency to some extent, they still require hours or days of interaction to learn skills. Recently, offline RL has been proposed as a framework for training RL policies on pre-existing datasets without any online interaction. However, constraining an algorithm to a fixed dataset induces a state-action distribution shift between training and inference, and limits its applicability to new tasks. In this work, we seek to get the best of both worlds: we consider the problem of pretraining a world model with offline data collected on a real robot, and then finetuning the model on online data collected by planning with the learned model. To mitigate extrapolation errors during online interaction, we propose to regularize the planner at test-time by balancing estimated returns and (epistemic) model uncertainty. We evaluate our method on a variety of visuo-motor control tasks in simulation and on a real robot, and find that our method enables few-shot finetuning to seen and unseen tasks even when offline data is limited. Videos, code, and data are available at https://yunhaifeng.com/FOWM .Comment: CoRL 2023 Oral; Project website: https://yunhaifeng.com/FOW

Cognitive deficits in obsessive compulsive disorder in tests which are sensitive to frontal lobe dysfunction

Forty patients with obsessive compulsive disorder (OCD) were compared to matched healthy controls on neuropsychological tests which are sensitive to frontal lobe dysfunction. On a computerised version of the Tower of London task of planning, OCD patients were no different to healthy controls in the accuracy of their solutions. There was no difference between the groups in the time spent thinking prior to making the first move or in the time spent thinking after the first move when "perfect move" solutions were considered. However, when the patients made a mistake, they spent more time than the controls generating alternative solutions or checking that the next move would be correct. The results suggest that OCD patients have a selective deficit in planning of generating alternative strategies when they make a mistake. In a separate attentional set-shifting task, OCD patients showed a continuous increase in terms of the number who failed at each stage of the task, including the crucial extra- dimensional set shifting stage. This suggests that OCD patients show deficits in both acquiring and maintaining cognitive sets. A sub-group of OCD patients who fail at or before the extra-dimensional shift stage also performed poorly on the Tower of London task. They are less accurate when solving problems and have a similar pattern of deficits to some neurosurgical patients with frontal lobe excisions. Both studies support the evidence of frontal-striatal dysfunction in OCD and the pattern of results is compared to that found in other known fronto-striatal disorders. The results are discussed in terms of a functional absence of a Supervisory Attentional System (Norman and Shallice, 1980)

Temporal dynamics of target selection and distractor suppression mechanisms in the right Frontal Eye Field

The ability of the human brain to selectively attend to relevant information while ignoring irrelevant distraction is essential for the successful completion of everyday tasks. The present PhD project aimed to unravel the temporal dynamics of target selection and distractor suppression in the Frontal Eye Field (FEF), a key node in the dorsolateral attention network. Previous research (Lega et al., 2019) had assessed the contribution of both IPS and FEF to the deployment of visuo- spatial attention by means of 10 Hz TMS during a visual search task. The stimulation was delivered in a post-stimulus epoch from 100 to 300 ms, considered crucial for attentional computations in visual search. This study found that the TMS protocol improved distractor suppression mechanisms, reducing the cost engendered by salient but task-irrelevant distractors. To further clarify the temporal contribution of right FEF to distractor suppression, two experiments were carried out. Experiment 1 applied single-pulse TMS over right FEF at three different time points, 50, 200 or 350 ms after search array onset. Experiment 2 aimed to exert a stronger TMS effect over right FEF while maintaining a temporal-punctate approach. It applied trains of triple-pulse TMS at 20 Hz over right FEF in three different time windows: from -50 to 50 ms (T1), from 100 to 200 ms (T2) and from 250 to 350 ms (T3) after the search array onset. While Experiment 1 showed only a general, time-unspecific and quasi- significant effect of stimulation over response times, Experiment 2 revealed that stimulation at T2 (100-200 ms) was associated with an increase of the distractor cost, specifically for distractors located contralaterally to the stimulation site. These findings support the role of right FEF in suppressing distractions from salient but irrelevant stimuli and suggest that TMS may activate/inhibit the neural network that regulates and limits interference from such distractions. Further research is needed to precisely assess the physiological effects of different TMS protocols of the right FEF and its influence on attentional computation

Affordances in Psychology, Neuroscience, and Robotics: A Survey

The concept of affordances appeared in psychology during the late 60s as an alternative perspective on the visual perception of the environment. It was revolutionary in the intuition that the way living beings perceive the world is deeply influenced by the actions they are able to perform. Then, across the last 40 years, it has influenced many applied fields, e.g., design, human-computer interaction, computer vision, and robotics. In this paper, we offer a multidisciplinary perspective on the notion of affordances. We first discuss the main definitions and formalizations of the affordance theory, then we report the most significant evidence in psychology and neuroscience that support it, and finally we review the most relevant applications of this concept in robotics

Multi-layer adaptation of group coordination in musical ensembles

Group coordination passes through an efficient integration of multimodal sources of information. This study examines complex non-verbal communication by recording movement kinematics from conductors and two sections of violinists of an orchestra adapting to a perturbation affecting their normal pattern of sensorimotor communication (rotation of half a turn of the first violinists\u2019 section). We show that different coordination signals are channeled through ancillary (head kinematics) and instrumental movements (bow kinematics). Each one of them affect coordination either at the inter-group or intra-group levels, therefore tapping into different modes of cooperation: complementary versus imitative coordination. Our study suggests that the co-regulation of group behavior is based on the exchange of information across several layers, each one of them tuned to carry specific coordinative signals. Multi-layer sensorimotor communication may be the key musicians and, more generally humans, use to flexibly communicate between each other in interactive sensorimotor tasks

Multi-layer adaptation of group coordination in musical ensembles

Group coordination passes through an efficient integration of multimodal sources of information. This study examines complex non-verbal communication by recording movement kinematics from conductors and two sections of violinists of an orchestra adapting to a perturbation affecting their normal pattern of sensorimotor communication (rotation of half a turn of the first violinists’ section). We show that different coordination signals are channeled through ancillary (head kinematics) and instrumental movements (bow kinematics). Each one of them affect coordination either at the inter-group or intra-group levels, therefore tapping into different modes of cooperation: complementary versus imitative coordination. Our study suggests that the co-regulation of group behavior is based on the exchange of information across several layers, each one of them tuned to carry specific coordinative signals. Multi-layer sensorimotor communication may be the key musicians and, more generally humans, use to flexibly communicate between each other in interactive sensorimotor tasks