2,943 research outputs found
Deep Dynamics Models for Learning Dexterous Manipulation
Dexterous multi-fingered hands can provide robots with the ability to
flexibly perform a wide range of manipulation skills. However, many of the more
complex behaviors are also notoriously difficult to control: Performing in-hand
object manipulation, executing finger gaits to move objects, and exhibiting
precise fine motor skills such as writing, all require finely balancing contact
forces, breaking and reestablishing contacts repeatedly, and maintaining
control of unactuated objects. Learning-based techniques provide the appealing
possibility of acquiring these skills directly from data, but current learning
approaches either require large amounts of data and produce task-specific
policies, or they have not yet been shown to scale up to more complex and
realistic tasks requiring fine motor skills. In this work, we demonstrate that
our method of online planning with deep dynamics models (PDDM) addresses both
of these limitations; we show that improvements in learned dynamics models,
together with improvements in online model-predictive control, can indeed
enable efficient and effective learning of flexible contact-rich dexterous
manipulation skills -- and that too, on a 24-DoF anthropomorphic hand in the
real world, using just 4 hours of purely real-world data to learn to
simultaneously coordinate multiple free-floating objects. Videos can be found
at https://sites.google.com/view/pddm/Comment: project website https://sites.google.com/view/pddm
Manipulation by Feel: Touch-Based Control with Deep Predictive Models
Touch sensing is widely acknowledged to be important for dexterous robotic
manipulation, but exploiting tactile sensing for continuous, non-prehensile
manipulation is challenging. General purpose control techniques that are able
to effectively leverage tactile sensing as well as accurate physics models of
contacts and forces remain largely elusive, and it is unclear how to even
specify a desired behavior in terms of tactile percepts. In this paper, we take
a step towards addressing these issues by combining high-resolution tactile
sensing with data-driven modeling using deep neural network dynamics models. We
propose deep tactile MPC, a framework for learning to perform tactile servoing
from raw tactile sensor inputs, without manual supervision. We show that this
method enables a robot equipped with a GelSight-style tactile sensor to
manipulate a ball, analog stick, and 20-sided die, learning from unsupervised
autonomous interaction and then using the learned tactile predictive model to
reposition each object to user-specified configurations, indicated by a goal
tactile reading. Videos, visualizations and the code are available here:
https://sites.google.com/view/deeptactilempcComment: Accepted to ICRA 201
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Dexterous multi-fingered robotic hands can perform a wide range of
manipulation skills, making them an appealing component for general-purpose
robotic manipulators. However, such hands pose a major challenge for autonomous
control, due to the high dimensionality of their configuration space and
complex intermittent contact interactions. In this work, we propose deep
reinforcement learning (deep RL) as a scalable solution for learning complex,
contact rich behaviors with multi-fingered hands. Deep RL provides an
end-to-end approach to directly map sensor readings to actions, without the
need for task specific models or policy classes. We show that contact-rich
manipulation behavior with multi-fingered hands can be learned by directly
training with model-free deep RL algorithms in the real world, with minimal
additional assumption and without the aid of simulation. We learn a variety of
complex behaviors on two different low-cost hardware platforms. We show that
each task can be learned entirely from scratch, and further study how the
learning process can be further accelerated by using a small number of human
demonstrations to bootstrap learning. Our experiments demonstrate that complex
multi-fingered manipulation skills can be learned in the real world in about
4-7 hours for most tasks, and that demonstrations can decrease this to 2-3
hours, indicating that direct deep RL training in the real world is a viable
and practical alternative to simulation and model-based control.
\url{https://sites.google.com/view/deeprl-handmanipulation}Comment: https://sites.google.com/view/deeprl-handmanipulatio
Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
Dexterous multi-fingered hands can accomplish fine manipulation behaviors
that are infeasible with simple robotic grippers. However, sophisticated
multi-fingered hands are often expensive and fragile. Low-cost soft hands offer
an appealing alternative to more conventional devices, but present considerable
challenges in sensing and actuation, making them difficult to apply to more
complex manipulation tasks. In this paper, we describe an approach to learning
from demonstration that can be used to train soft robotic hands to perform
dexterous manipulation tasks. Our method uses object-centric demonstrations,
where a human demonstrates the desired motion of manipulated objects with their
own hands, and the robot autonomously learns to imitate these demonstrations
using reinforcement learning. We propose a novel algorithm that allows us to
blend and select a subset of the most feasible demonstrations to learn to
imitate on the hardware, which we use with an extension of the guided policy
search framework to use multiple demonstrations to learn generalizable neural
network policies. We demonstrate our approach on the RBO Hand 2, with learned
motor skills for turning a valve, manipulating an abacus, and grasping.Comment: Accepted at International Conference on Intelligent Robots and
Systems(IROS) 2016. Pdf file updated for stylistic consistenc
Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning
Robots must know how to be gentle when they need to interact with fragile
objects, or when the robot itself is prone to wear and tear. We propose an
approach that enables deep reinforcement learning to train policies that are
gentle, both during exploration and task execution. In a reward-based learning
environment, a natural approach involves augmenting the (task) reward with a
penalty for non-gentleness, which can be defined as excessive impact force.
However, augmenting with only this penalty impairs learning: policies get stuck
in a local optimum which avoids all contact with the environment. Prior
research has shown that combining auxiliary tasks or intrinsic rewards can be
beneficial for stabilizing and accelerating learning in sparse-reward domains,
and indeed we find that introducing a surprise-based intrinsic reward does
avoid the no-contact failure case. However, we show that a simple
dynamics-based surprise is not as effective as penalty-based surprise.
Penalty-based surprise, based on predicting forceful contacts, has a further
benefit: it encourages exploration which is contact-rich yet gentle. We
demonstrate the effectiveness of the approach using a complex, tendon-powered
robot hand with tactile sensors. Videos are available at
http://sites.google.com/view/gentlemanipulation
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Despite decades of research, general purpose in-hand manipulation remains one
of the unsolved challenges of robotics. One of the contributing factors that
limit current robotic manipulation systems is the difficulty of precisely
sensing contact forces -- sensing and reasoning about contact forces are
crucial to accurately control interactions with the environment. As a step
towards enabling better robotic manipulation, we introduce DIGIT, an
inexpensive, compact, and high-resolution tactile sensor geared towards in-hand
manipulation. DIGIT improves upon past vision-based tactile sensors by
miniaturizing the form factor to be mountable on multi-fingered hands, and by
providing several design improvements that result in an easier, more repeatable
manufacturing process, and enhanced reliability. We demonstrate the
capabilities of the DIGIT sensor by training deep neural network model-based
controllers to manipulate glass marbles in-hand with a multi-finger robotic
hand. To provide the robotic community access to reliable and low-cost tactile
sensors, we open-source the DIGIT design at https://digit.ml/.Comment: 8 pages, published in the IEEE Robotics and Automation Letters (RA-L
Let's Push Things Forward: A Survey on Robot Pushing
As robot make their way out of factories into human environments, outer
space, and beyond, they require the skill to manipulate their environment in
multifarious, unforeseeable circumstances. With this regard, pushing is an
essential motion primitive that dramatically extends a robot's manipulation
repertoire. In this work, we review the robotic pushing literature. While
focusing on work concerned with predicting the motion of pushed objects, we
also cover relevant applications of pushing for planning and control. Beginning
with analytical approaches, under which we also subsume physics engines, we
then proceed to discuss work on learning models from data. In doing so, we
dedicate a separate section to deep learning approaches which have seen a
recent upsurge in the literature. Concluding remarks and further research
perspectives are given at the end of the paper
Leveraging Contact Forces for Learning to Grasp
Grasping objects under uncertainty remains an open problem in robotics
research. This uncertainty is often due to noisy or partial observations of the
object pose or shape. To enable a robot to react appropriately to unforeseen
effects, it is crucial that it continuously takes sensor feedback into account.
While visual feedback is important for inferring a grasp pose and reaching for
an object, contact feedback offers valuable information during manipulation and
grasp acquisition. In this paper, we use model-free deep reinforcement learning
to synthesize control policies that exploit contact sensing to generate robust
grasping under uncertainty. We demonstrate our approach on a multi-fingered
hand that exhibits more complex finger coordination than the commonly used
two-fingered grippers. We conduct extensive experiments in order to assess the
performance of the learned policies, with and without contact sensing. While it
is possible to learn grasping policies without contact sensing, our results
suggest that contact feedback allows for a significant improvement of grasping
robustness under object pose uncertainty and for objects with a complex shape.Comment: 7 pages, 5 figures, Submitted to ICRA'1
Reinforcement and Imitation Learning for Diverse Visuomotor Skills
We propose a model-free deep reinforcement learning method that leverages a
small amount of demonstration data to assist a reinforcement learning agent. We
apply this approach to robotic manipulation tasks and train end-to-end
visuomotor policies that map directly from RGB camera inputs to joint
velocities. We demonstrate that our approach can solve a wide variety of
visuomotor tasks, for which engineering a scripted controller would be
laborious. In experiments, our reinforcement and imitation agent achieves
significantly better performances than agents trained with reinforcement
learning or imitation learning alone. We also illustrate that these policies,
trained with large visual and dynamics variations, can achieve preliminary
successes in zero-shot sim2real transfer. A brief visual description of this
work can be viewed in https://youtu.be/EDl8SQUNjj0Comment: 13 pages, 6 figures, Published in RSS 201
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
We propose a plan online and learn offline (POLO) framework for the setting
where an agent, with an internal model, needs to continually act and learn in
the world. Our work builds on the synergistic relationship between local
model-based control, global value function learning, and exploration. We study
how local trajectory optimization can cope with approximation errors in the
value function, and can stabilize and accelerate value function learning.
Conversely, we also study how approximate value functions can help reduce the
planning horizon and allow for better policies beyond local solutions. Finally,
we also demonstrate how trajectory optimization can be used to perform
temporally coordinated exploration in conjunction with estimating uncertainty
in value function approximation. This exploration is critical for fast and
stable learning of the value function. Combining these components enable
solutions to complex simulated control tasks, like humanoid locomotion and
dexterous in-hand manipulation, in the equivalent of a few minutes of
experience in the real world.Comment: The first two authors contributed equally. Accepted at ICLR 2019.
Supplementary videos available at: https://sites.google.com/view/polo-mp
- …