1,095 research outputs found
Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation
Manipulation of deformable objects, such as ropes and cloth, is an important
but challenging problem in robotics. We present a learning-based system where a
robot takes as input a sequence of images of a human manipulating a rope from
an initial to goal configuration, and outputs a sequence of actions that can
reproduce the human demonstration, using only monocular images as input. To
perform this task, the robot learns a pixel-level inverse dynamics model of
rope manipulation directly from images in a self-supervised manner, using about
60K interactions with the rope collected autonomously by the robot. The human
demonstration provides a high-level plan of what to do and the low-level
inverse model is used to execute the plan. We show that by combining the high
and low-level plans, the robot can successfully manipulate a rope into a
variety of target shapes using only a sequence of human-provided images for
direction.Comment: 8 pages, accepted to International Conference on Robotics and
Automation (ICRA) 201
Multiform Adaptive Robot Skill Learning from Humans
Object manipulation is a basic element in everyday human lives. Robotic
manipulation has progressed from maneuvering single-rigid-body objects with
firm grasping to maneuvering soft objects and handling contact-rich actions.
Meanwhile, technologies such as robot learning from demonstration have enabled
humans to intuitively train robots. This paper discusses a new level of robotic
learning-based manipulation. In contrast to the single form of learning from
demonstration, we propose a multiform learning approach that integrates
additional forms of skill acquisition, including adaptive learning from
definition and evaluation. Moreover, going beyond state-of-the-art technologies
of handling purely rigid or soft objects in a pseudo-static manner, our work
allows robots to learn to handle partly rigid partly soft objects with
time-critical skills and sophisticated contact control. Such capability of
robotic manipulation offers a variety of new possibilities in human-robot
interaction.Comment: Accepted to 2017 Dynamic Systems and Control Conference (DSCC),
Tysons Corner, VA, October 11-1
Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation
Understanding and manipulating deformable objects (e.g., ropes and fabrics)
is an essential yet challenging task with broad applications. Difficulties come
from complex states and dynamics, diverse configurations and high-dimensional
action space of deformable objects. Besides, the manipulation tasks usually
require multiple steps to accomplish, and greedy policies may easily lead to
local optimal states. Existing studies usually tackle this problem using
reinforcement learning or imitating expert demonstrations, with limitations in
modeling complex states or requiring hand-crafted expert policies. In this
paper, we study deformable object manipulation using dense visual affordance,
with generalization towards diverse states, and propose a novel kind of
foresightful dense affordance, which avoids local optima by estimating states'
values for long-term manipulation. We propose a framework for learning this
representation, with novel designs such as multi-stage stable learning and
efficient self-supervised data collection without experts. Experiments
demonstrate the superiority of our proposed foresightful dense affordance.
Project page: https://hyperplane-lab.github.io/DeformableAffordanc
GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policy
Due to the inherent uncertainty in their deformability during motion,
previous methods in deformable object manipulation, such as rope and cloth,
often required hundreds of real-world demonstrations to train a manipulation
policy for each object, which hinders their applications in our ever-changing
world. To address this issue, we introduce GenDOM, a framework that allows the
manipulation policy to handle different deformable objects with only a single
real-world demonstration. To achieve this, we augment the policy by
conditioning it on deformable object parameters and training it with a diverse
range of simulated deformable objects so that the policy can adjust actions
based on different object parameters. At the time of inference, given a new
object, GenDOM can estimate the deformable object parameters with only a single
real-world demonstration by minimizing the disparity between the grid density
of point clouds of real-world demonstrations and simulations in a
differentiable physics simulator. Empirical validations on both simulated and
real-world object manipulation setups clearly show that our method can
manipulate different objects with a single demonstration and significantly
outperforms the baseline in both environments (a 62% improvement for in-domain
ropes and a 15% improvement for out-of-distribution ropes in simulation, as
well as a 26% improvement for ropes and a 50% improvement for cloths in the
real world), demonstrating the effectiveness of our approach in one-shot
deformable object manipulation.Comment: Extended version of arXiv:2306.0987
- …