Uncertainty-driven Affordance Discovery for Efficient Robotics Manipulation

Robotics affordances, providing information about what actions can be taken in a given situation, can aid robotics manipulation. However, learning about affordances requires expensive large annotated datasets of interactions or demonstrations. In this work, we show active learning can mitigate this problem and propose the use of uncertainty to drive an interactive affordance discovery process. We show that our method enables the efficient discovery of visual affordances for several action primitives, such as grasping, stacking objects, or opening drawers, strongly improving data efficiency and allowing us to learn grasping affordances on a real-world setup with an xArm 6 robot arm in a small number of trials.Comment: Presented at the GMPL workshop @ RSS 202

Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects

Articulated object manipulation is a fundamental yet challenging task in robotics. Due to significant geometric and semantic variations across object categories, previous manipulation models struggle to generalize to novel categories. Few-shot learning is a promising solution for alleviating this issue by allowing robots to perform a few interactions with unseen objects. However, extant approaches often necessitate costly and inefficient test-time interactions with each unseen instance. Recognizing this limitation, we observe that despite their distinct shapes, different categories often share similar local geometries essential for manipulation, such as pullable handles and graspable edges - a factor typically underutilized in previous few-shot learning works. To harness this commonality, we introduce 'Where2Explore', an affordance learning framework that effectively explores novel categories with minimal interactions on a limited number of instances. Our framework explicitly estimates the geometric similarity across different categories, identifying local areas that differ from shapes in the training categories for efficient exploration while concurrently transferring affordance knowledge to similar parts of the objects. Extensive experiments in simulated and real-world environments demonstrate our framework's capacity for efficient few-shot exploration and generalization

Self-Supervised Learning of Action Affordances as Interaction Modes

When humans perform a task with an articulated object, they interact with the object only in a handful of ways, while the space of all possible interactions is nearly endless. This is because humans have prior knowledge about what interactions are likely to be successful, i.e., to open a new door we first try the handle. While learning such priors without supervision is easy for humans, it is notoriously hard for machines. In this work, we tackle unsupervised learning of priors of useful interactions with articulated objects, which we call interaction modes. In contrast to the prior art, we use no supervision or privileged information; we only assume access to the depth sensor in the simulator to learn the interaction modes. More precisely, we define a successful interaction as the one changing the visual environment substantially and learn a generative model of such interactions, that can be conditioned on the desired goal state of the object. In our experiments, we show that our model covers most of the human interaction modes, outperforms existing state-of-the-art methods for affordance learning, and can generalize to objects never seen during training. Additionally, we show promising results in the goal-conditional setup, where our model can be quickly fine-tuned to perform a given task. We show in the experiments that such affordance learning predicts interaction which covers most modes of interaction for the querying articulated object and can be fine-tuned to a goal-conditional model. For supplementary: https://actaim.github.io

Affordances in Psychology, Neuroscience, and Robotics: A Survey

The concept of affordances appeared in psychology during the late 60s as an alternative perspective on the visual perception of the environment. It was revolutionary in the intuition that the way living beings perceive the world is deeply influenced by the actions they are able to perform. Then, across the last 40 years, it has influenced many applied fields, e.g., design, human-computer interaction, computer vision, and robotics. In this paper, we offer a multidisciplinary perspective on the notion of affordances. We first discuss the main definitions and formalizations of the affordance theory, then we report the most significant evidence in psychology and neuroscience that support it, and finally we review the most relevant applications of this concept in robotics

Europa Universalis IV and Fairly Radical History(ing)

Tässä tutkimuksessa tarkastellaan Europa Universalis IV-strategiapeliä historioimisen (history(ing)) kannalta kolonialismin kautta. Työssä määritellään peli historiana, jonka lisäksi arvioidaan pelaajien osallistumista ”suhteellisen radikaaliin historioimiseen” pelin yhteisöfoorumeilla. Lähdemateriaali koostuu siis itse pelistä ja sen lisäosista sekä tekstuaalisesta osasta valikoitujen pelin kehittämistä koskevien keskustelujen muodossa. Työssä tutkitaan pelin kehittäjä-historioitsijan ja pelaaja-historioitsijan (developer-historian ja player-historian) historioimisen tasoa ja mahdollista kohtaamista em. keskusteluissa, jossa he ottaisivat osaa dialogiseen ja dialektiseen historioimiseen. Tutkielman metodologia rakentui yhdistämällä kaksi käsitteellistä tasoa: historiallinen-tarina-pelitila ((hi)story-play-space; Chapman 2016) ja ongelmatila (problem space; McCall 2012). Tämä mahdollisti sekä pelin mekaanisen retoriikan (procedural rhetoric) ja sen tekemien historiallisten väittämien avaamisen että myös syvällisen analyysin siitä, minkälaista historiaa peli pelaajille tarjoaa. Tekstuaalisen osion laadullinen analyysi keskittyi keskustelun tasoon molempien näiden näkökulmien osalta. Pelin laajuuden vuoksi materiaalia rajaamaan valittiin esimerkkitapaukseksi kolonialismi, jonka voi nähdä yhtenä tärkeimmistä pelimekaniikoista. Tutkielman tuloksena näyttää siltä, että yhteisöfoorumeilla todella osallistutaan ”suhteellisen radikaaliin historioimiseen”, vaikka kyseessä onkin vain pieni osa kaikista pelaajista. Keskustelussa käytettyjä väittämiä perusteltiin ajoittain tutkimuskirjallisuuden avulla, eikä keskustelu päättynyt historiallisiin aiheisiin, vaan jatkui itse pelin pohtimiseen historiallisen kerronnan muotona. Mikäli lähdemateriaalia olisi valittu yhteisöfoorumeilta tarkemmin, olisi saattanut olla mahdollista arvioida osallistumista historioimiseen vielä kattavammin. Lisäksi tässä materiaalissa pelin kehittäjän ja pelaajan välinen dialogi jäi todella vähäiseksi ja mahdollinen tuleva tutkimus voisi keskittyä tutkimaan nimenomaan kysymyksiä siitä, miten pelin muutettavat seikat valitaan ja ovatko muutokset jotenkin neuvoteltuja kehittäjän ja pelaajien välillä.This thesis defined the grand strategy game Europa Universalis IV as a historical piece, as well as looked into the engagement of players into fairly radical history(ing) on the community forums. The primary material for analysis consisted of the game itself with its expansions, along with a textual component in the form of selected Developer Diary entries. The aim was to explore the level of history(ing) by developer-historians and player-historians alike, and their possible confrontations in these Diaries, i.e., the engagement in dialogic and dialectical process of history(ing). The methodology was synthetized by merging two conceptualizations: (hi)story-play-space (Chapman 2016) and problem space (McCall 2012). This allowed not only to look at the procedural rhetoric of EUIV and the claims made about history in the game, but also at what kind of history it afforded to the players. The qualitative analysis for the text component concentrated on the level of discussion on both of these aspects. Colonialism provided a case study for analysis since it has central gameplay mechanics. In essence, the results of this study found that there certainly is some ”fairly radical history(ing)” (Chapman 2016) going on in the community forums, albeit by a small minority of the players. The discussion was sometimes backed by actual sources, and not only was history debated, but also the game form itself in respect to history. With a more carefully planned selection of primary material from the community forums, it would have been possible to estimate the actual level of participation, which was not possible with the current data. Furthermore, there was very little dialogue between the Developer and the player-base, and more research should go into how the mechanics to be altered are chosen, and whether these changes are negotiated or not

Discovering Affordances Through Perception and Manipulation

International audienceConsidering perception as an observation process only is the very reason for which robotic perception methods are to date unable to provide a general capacity of scene understanding. Related work in neuroscience has shown that there is a strong relationship between perception and action. We believe that considering perception in relation to action requires to interpret the scene in terms of the agent's own potential capabilities. In this paper, we propose a Bayesian approach for learning sensorimotor representations through the interaction between action and observation capabilities. We represent the notion of affordance as a probabilistic relation between three elements: objects, actions and effects. Experiments for affordances discovery were performed on a real robotic platform in an unsupervised way assuming a limited set of innate capabilities. Results show dependency relations that connect the three elements in a common frame: affordances. The increasing number of interactions and observations results in a Bayesian network that captures the relationships between them. The learned representation can be used for prediction tasks

Identification of Invariant Sensorimotor Structures as a Prerequisite for the Discovery of Objects

Perceiving the surrounding environment in terms of objects is useful for any general purpose intelligent agent. In this paper, we investigate a fundamental mechanism making object perception possible, namely the identification of spatio-temporally invariant structures in the sensorimotor experience of an agent. We take inspiration from the Sensorimotor Contingencies Theory to define a computational model of this mechanism through a sensorimotor, unsupervised and predictive approach. Our model is based on processing the unsupervised interaction of an artificial agent with its environment. We show how spatio-temporally invariant structures in the environment induce regularities in the sensorimotor experience of an agent, and how this agent, while building a predictive model of its sensorimotor experience, can capture them as densely connected subgraphs in a graph of sensory states connected by motor commands. Our approach is focused on elementary mechanisms, and is illustrated with a set of simple experiments in which an agent interacts with an environment. We show how the agent can build an internal model of moving but spatio-temporally invariant structures by performing a Spectral Clustering of the graph modeling its overall sensorimotor experiences. We systematically examine properties of the model, shedding light more globally on the specificities of the paradigm with respect to methods based on the supervised processing of collections of static images.Comment: 24 pages, 10 figures, published in Frontiers Robotics and A