82 research outputs found
DMFC-GraspNet: Differentiable Multi-Fingered Robotic Grasp Generation in Cluttered Scenes
Robotic grasping is a fundamental skill required for object manipulation in
robotics. Multi-fingered robotic hands, which mimic the structure of the human
hand, can potentially perform complex object manipulation. Nevertheless,
current techniques for multi-fingered robotic grasping frequently predict only
a single grasp for each inference time, limiting computational efficiency and
their versatility, i.e. unimodal grasp distribution. This paper proposes a
differentiable multi-fingered grasp generation network (DMFC-GraspNet) with
three main contributions to address this challenge. Firstly, a novel neural
grasp planner is proposed, which predicts a new grasp representation to enable
versatile and dense grasp predictions. Secondly, a scene creation and label
mapping method is developed for dense labeling of multi-fingered robotic hands,
which allows a dense association of ground truth grasps. Thirdly, we propose to
train DMFC-GraspNet end-to-end using using a forward-backward automatic
differentiation approach with both a supervised loss and a differentiable
collision loss and a generalized Q 1 grasp metric loss. The proposed approach
is evaluated using the Shadow Dexterous Hand on Mujoco simulation and ablated
by different choices of loss functions. The results demonstrate the
effectiveness of the proposed approach in predicting versatile and dense
grasps, and in advancing the field of multi-fingered robotic grasping.Comment: Submitted IROS 2023 workshop "Policy Learning in Geometric Spaces
DVGG: Deep Variational Grasp Generation for Dextrous Manipulation
Grasping with anthropomorphic robotic hands involves much more hand-object
interactions compared to parallel-jaw grippers. Modeling hand-object
interactions is essential to the study of multi-finger hand dextrous
manipulation. This work presents DVGG, an efficient grasp generation network
that takes single-view observation as input and predicts high-quality grasp
configurations for unknown objects. In general, our generative model consists
of three components: 1) Point cloud completion for the target object based on
the partial observation; 2) Diverse sets of grasps generation given the
complete point cloud; 3) Iterative grasp pose refinement for physically
plausible grasp optimization. To train our model, we build a large-scale
grasping dataset that contains about 300 common object models with 1.5M
annotated grasps in simulation. Experiments in simulation show that our model
can predict robust grasp poses with a wide variety and high success rate. Real
robot platform experiments demonstrate that the model trained on our dataset
performs well in the real world. Remarkably, our method achieves a grasp
success rate of 70.7\% for novel objects in the real robot platform, which is a
significant improvement over the baseline methods.Comment: Accepted by Robotics and Automation Letters (RA-L, 2021
Multi-FinGAN: generative coarse-to-fine sampling of multi-finger grasps
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksWhile there exists many methods for manipulating rigid objects with parallel-jaw grippers, grasping with multi- finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we present Multi-FinGAN, a fast generative multi-finger grasp sampling method that synthesizes high quality grasps directly from RGB-D images in about a second. We achieve this by training in an end-to-end fashion a coarse-to-fine model composed of a classification network that distinguishes grasp types according to a specific taxonomy and a refinement network that produces refined grasp poses and joint angles. We experimentally validate and benchmark our method against a standard grasp-sampling method on 790 grasps in simulation and 20 grasps on a real Franka Emika Panda. All experimental results using our method show consistent improvements both in terms of grasp quality metrics and grasp success rate. Remarkably, our approach is up to 20-30 times faster than the baseline, a significant improvement that opens the door to feedback-based grasp re-planning and task informative grasping. Code is available at https://irobotics.aalto.fi/multi-fingan/.Peer ReviewedPostprint (author's final draft
Geometry Matching for Multi-Embodiment Grasping
Many existing learning-based grasping approaches concentrate on a single
embodiment, provide limited generalization to higher DoF end-effectors and
cannot capture a diverse set of grasp modes. We tackle the problem of grasping
using multiple embodiments by learning rich geometric representations for both
objects and end-effectors using Graph Neural Networks. Our novel method -
GeoMatch - applies supervised learning on grasping data from multiple
embodiments, learning end-to-end contact point likelihood maps as well as
conditional autoregressive predictions of grasps keypoint-by-keypoint. We
compare our method against baselines that support multiple embodiments. Our
approach performs better across three end-effectors, while also producing
diverse grasps. Examples, including real robot demos, can be found at
geo-match.github.io
Deep Learning Approaches to Grasp Synthesis: A Review
Grasping is the process of picking up an object by applying forces and torques at a set of contacts. Recent advances in deep learning methods have allowed rapid progress in robotic object grasping. In this systematic review, we surveyed the publications over the last decade, with a particular interest in grasping an object using all six degrees of freedom of the end-effector pose. Our review found four common methodologies for robotic grasping: sampling-based approaches, direct regression, reinforcement learning, and exemplar approaches In addition, we found two “supporting methods” around grasping that use deep learning to support the grasping process, shape approximation, and affordances. We have distilled the publications found in this systematic review (85 papers) into ten key takeaways we consider crucial for future robotic grasping and manipulation research
Intersection-free Robot Manipulation with Soft-Rigid Coupled Incremental Potential Contact
This paper presents a novel simulation platform, ZeMa, designed for robotic
manipulation tasks concerning soft objects. Such simulation ideally requires
three properties: two-way soft-rigid coupling, intersection-free guarantees,
and frictional contact modeling, with acceptable runtime suitable for deep and
reinforcement learning tasks. Current simulators often satisfy only a subset of
these needs, primarily focusing on distinct rigid-rigid or soft-soft
interactions. The proposed ZeMa prioritizes physical accuracy and integrates
the incremental potential contact method, offering unified dynamics simulation
for both soft and rigid objects. It efficiently manages soft-rigid contact,
operating 75x faster than baseline tools with similar methodologies like
IPC-GraspSim. To demonstrate its applicability, we employ it for parallel grasp
generation, penetrated grasp repair, and reinforcement learning for grasping,
successfully transferring the trained RL policy to real-world scenarios
UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy
In this work, we tackle the problem of learning universal robotic dexterous
grasping from a point cloud observation under a table-top setting. The goal is
to grasp and lift up objects in high-quality and diverse ways and generalize
across hundreds of categories and even the unseen. Inspired by successful
pipelines used in parallel gripper grasping, we split the task into two stages:
1) grasp proposal (pose) generation and 2) goal-conditioned grasp execution.
For the first stage, we propose a novel probabilistic model of grasp pose
conditioned on the point cloud observation that factorizes rotation from
translation and articulation. Trained on our synthesized large-scale dexterous
grasp dataset, this model enables us to sample diverse and high-quality
dexterous grasp poses for the object point cloud.For the second stage, we
propose to replace the motion planning used in parallel gripper grasping with a
goal-conditioned grasp policy, due to the complexity involved in dexterous
grasping execution. Note that it is very challenging to learn this highly
generalizable grasp policy that only takes realistic inputs without oracle
states. We thus propose several important innovations, including state
canonicalization, object curriculum, and teacher-student distillation.
Integrating the two stages, our final pipeline becomes the first to achieve
universal generalization for dexterous grasping, demonstrating an average
success rate of more than 60\% on thousands of object instances, which
significantly outperforms all baselines, meanwhile showing only a minimal
generalization gap.Comment: Accepted to CVPR 202
Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
Grasping objects with limited or no prior knowledge about them is a highly
relevant skill in assistive robotics. Still, in this general setting, it has
remained an open problem, especially when it comes to only partial
observability and versatile grasping with multi-fingered hands. We present a
novel, fast, and high fidelity deep learning pipeline consisting of a shape
completion module that is based on a single depth image, and followed by a
grasp predictor that is based on the predicted object shape. The shape
completion network is based on VQDIF and predicts spatial occupancy values at
arbitrary query points. As grasp predictor, we use our two-stage architecture
that first generates hand poses using an autoregressive model and then
regresses finger joint configurations per pose. Critical factors turn out to be
sufficient data realism and augmentation, as well as special attention to
difficult cases during training. Experiments on a physical robot platform
demonstrate successful grasping of a wide range of household objects based on a
depth image from a single viewpoint. The whole pipeline is fast, taking only
about 1 s for completing the object's shape (0.7 s) and generating 1000 grasps
(0.3 s).Comment: 8 pages, 10 figures, 3 tables, 1 algorithm, 2023 IEEE-RAS
International Conference on Humanoid Robots (Humanoids), Project page:
https://dlr-alr.github.io/2023-humanoids-completio
- …