Object and Relation Centric Representations for Push Effect Prediction

Pushing is an essential non-prehensile manipulation skill used for tasks ranging from pre-grasp manipulation to scene rearrangement, reasoning about object relations in the scene, and thus pushing actions have been widely studied in robotics. The effective use of pushing actions often requires an understanding of the dynamics of the manipulated objects and adaptation to the discrepancies between prediction and reality. For this reason, effect prediction and parameter estimation with pushing actions have been heavily investigated in the literature. However, current approaches are limited because they either model systems with a fixed number of objects or use image-based representations whose outputs are not very interpretable and quickly accumulate errors. In this paper, we propose a graph neural network based framework for effect prediction and parameter estimation of pushing actions by modeling object relations based on contacts or articulations. Our framework is validated both in real and simulated environments containing different shaped multi-part objects connected via different types of joints and objects with different masses. Our approach enables the robot to predict and adapt the effect of a pushing action as it observes the scene. Further, we demonstrate 6D effect prediction in the lever-up action in the context of robot-based hard-disk disassembly.Comment: Project Page: https://fzaero.github.io/push_learning

Versatile Multi-Contact Planning and Control for Legged Loco-Manipulation

Loco-manipulation planning skills are pivotal for expanding the utility of robots in everyday environments. These skills can be assessed based on a system's ability to coordinate complex holistic movements and multiple contact interactions when solving different tasks. However, existing approaches have been merely able to shape such behaviors with hand-crafted state machines, densely engineered rewards, or pre-recorded expert demonstrations. Here, we propose a minimally-guided framework that automatically discovers whole-body trajectories jointly with contact schedules for solving general loco-manipulation tasks in pre-modeled environments. The key insight is that multi-modal problems of this nature can be formulated and treated within the context of integrated Task and Motion Planning (TAMP). An effective bilevel search strategy is achieved by incorporating domain-specific rules and adequately combining the strengths of different planning techniques: trajectory optimization and informed graph search coupled with sampling-based planning. We showcase emergent behaviors for a quadrupedal mobile manipulator exploiting both prehensile and non-prehensile interactions to perform real-world tasks such as opening/closing heavy dishwashers and traversing spring-loaded doors. These behaviors are also deployed on the real system using a two-layer whole-body tracking controller

Planning Robust Strategies for Constructing Multi-object Arrangements

A crucial challenge in robotics is achieving reliable results in spite of sensing and control uncertainty. A prominent strategy for dealing with uncertainty is to construct a feedback policy, where actions are chosen as a function of the current state estimate. However, constructing such policies is computationally very difficult. An alternative strategy is conformant planning which finds open-loop action sequences that achieve the goal for all input states and action outcomes. In this work, we investigate the conformant planning approach to robot manipulation. In particular, we tackle the problem of pushing multiple objects simultaneously to achieve a specified arrangement. Conformant planning is a belief-state planning problem. A belief state is the set of all possible states of the world, and the goal is to find a sequence of actions that will bring an initial belief state to a goal belief state To do forward belief-state planning, we created a deterministic belief-state transition model from supervised learning based on physics simulations. A key pitfall in conformant planning is that the complexity of the belief state tends to increase with each operation, making it increasingly harder to compute the effect of actions. This work explores the idea that we can construct conformant plans for robot manipulation by only using actions resulting in compact belief states

Object Isolation With Minimal Impact Towards The Object Of Interest In A Complex Environment Using Manipulation Primitives

It is common in the field of robotic manipulation to specifically target and precisely move, displace or manipulate the targeted object of interest. This however may not always be the best possible course of action as there are situations where it is not possible to manipulate the object of interest or is not in a condition to be manipulated. This research paper explores and subsequently proposes 3 object isolation technique for the purpose of isolating a targeted object of interest from the environment incontras to the standard utilization of object singulation technique. The goal was to develop an algorithm than can successfully isolate the object of interest from the environment via removing the environment without/with minimal impact towards the object of interest. Results from the experiment indicated that the proposed algorithms can successfully isolate the object of interest with minimal impact towards the object of interest scoring an average of 0.85cm/actuation for MSMAPPS, 0.75cm/actuation for MSMAPOS and finally 0.27cm/actuation for BSMAPOS. These results indicates a relatively small displacement per actuation at 4.35% displacement per actuation, 3.75% displacement per actuation, and 1.35% displacement per actuation relative to the workspace respectivel

Efficient Object Isolation In Complex Environment Using Manipulation Primitive On A Vision Based Mobile 6DOF Robotic Arm

This paper explores the idea of manipulation aided- perception in the context of isolating an object of interest from other small objects of varying degree of clusterization in order to obtain high quality training images.The robot utilizes a novel algorithm to plot out the position for each noise objects and its destined position as well as its trajectory and then utilizes manipulation primitives (pushing motion) to move said object along the planned trajectory.The method was demonstrated using Vrep simulation software which simulated a Kuka YouBot fitted with a camera on the gripper.We evaluated our approach by simulating the robot manipulators in an experiment which successfully isolate the object of interest from noise objects with at a rate of 77.46% at an average of 0.56 manipulations per object compared to others at 1.76 manipulations subsequently speeding up the time taken for manipulation from 12.58 minutes to 2.6 minutes however suffers from a tradeoff in terms of accuracy when comparing the similar works to our proposed method

Concepts in Action

This open access book is a timely contribution in presenting recent issues, approaches, and results that are not only central to the highly interdisciplinary field of concept research but also particularly important to newly emergent paradigms and challenges. The contributors present a unique, holistic picture for the understanding and use of concepts from a wide range of fields including cognitive science, linguistics, philosophy, psychology, artificial intelligence, and computer science. The chapters focus on three distinct points of view that lie at the core of concept research: representation, learning, and application. The contributions present a combination of theoretical, experimental, computational, and applied methods that appeal to students and researchers working in these fields