Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation

We develop an approach that benefits from large simulated datasets and takes full advantage of the limited online data that is most relevant. We propose a variant of Bayesian optimization that alternates between using informed and uninformed kernels. With this Bernoulli Alternation Kernel we ensure that discrepancies between simulation and reality do not hinder adapting robot control policies online. The proposed approach is applied to a challenging real-world problem of task-oriented grasping with novel objects. Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores. We learn task scores from a labeled dataset with a convolutional network, which is used to construct an informed kernel for our variant of Bayesian optimization. Experiments on an ABB Yumi robot with real sensor data demonstrate success of our approach, despite the challenge of fulfilling task requirements and high uncertainty over physical properties of objects.Comment: To appear in 2nd Conference on Robot Learning (CoRL) 201

Grasp plannind under task-specific contact constraints

Several aspects have to be addressed before realizing the dream of a robotic hand-arm system with human-like capabilities, ranging from the consolidation of a proper mechatronic design, to the development of precise, lightweight sensors and actuators, to the efficient planning and control of the articular forces and motions required for interaction with the environment. This thesis provides solution algorithms for a main problem within the latter aspect, known as the {\em grasp planning} problem: Given a robotic system formed by a multifinger hand attached to an arm, and an object to be grasped, both with a known geometry and location in 3-space, determine how the hand-arm system should be moved without colliding with itself or with the environment, in order to firmly grasp the object in a suitable way. Central to our algorithms is the explicit consideration of a given set of hand-object contact constraints to be satisfied in the final grasp configuration, imposed by the particular manipulation task to be performed with the object. This is a distinguishing feature from other grasp planning algorithms given in the literature, where a means of ensuring precise hand-object contact locations in the resulting grasp is usually not provided. These conventional algorithms are fast, and nicely suited for planning grasps for pick-an-place operations with the object, but not for planning grasps required for a specific manipulation of the object, like those necessary for holding a pen, a pair of scissors, or a jeweler's screwdriver, for instance, when writing, cutting a paper, or turning a screw, respectively. To be able to generate such highly-selective grasps, we assume that a number of surface regions on the hand are to be placed in contact with a number of corresponding regions on the object, and enforce the fulfilment of such constraints on the obtained solutions from the very beginning, in addition to the usual constraints of grasp restrainability, manipulability and collision avoidance. The proposed algorithms can be applied to robotic hands of arbitrary structure, possibly considering compliance in the joints and the contacts if desired, and they can accommodate general patch-patch contact constraints, instead of more restrictive contact types occasionally considered in the literature. It is worth noting, also, that while common force-closure or manipulability indices are used to asses the quality of grasps, no particular assumption is made on the mathematical properties of the quality index to be used, so that any quality criterion can be accommodated in principle. The algorithms have been tested and validated on numerous situations involving real mechanical hands and typical objects, and find applications in classical or emerging contexts like service robotics, telemedicine, space exploration, prosthetics, manipulation in hazardous environments, or human-robot interaction in general

A compliance-centric view of grasping

We advocate the central importance of compliance for grasp performance and demonstrate that grasp algorithms can achieve robust performance by explicitly considering and exploiting mechanical compliance of the grasping hand. Specifically, we consider the problem of robust grasping in the absence of a priori object models, focusing on object capture and grasp stability under variations of object shape for a given robotic hand. We present a simple characterization of the relationship between hand compliance, object shape, and grasp success. Based on this hypothesis, we devise a compliance-centric grasping algorithm. Real-world experiments show that this algorithm outperforms compliance-agnostic grasping, eliminates the need for explicit contact state planning, and simplifies the perceptual requirements when no a priori information about the environment is available.EC/FP7/248258/EU/Flexible Skill Acquisition and Intuitive Robot Tasking for Mobile Manipulation in the Real World/FIRST-M

Characterisation of Grasp Quality Metrics

Robot grasp quality metrics are used to evaluate, compare and select robotic grasp configurations. Many of them have been proposed based on a diversity of underlying principles and to assess different aspects of the grasp configurations. As a consequence, some of them provide similar information but other can provide completely different assessments. Combinations of metrics have been proposed in order to provide global indexes, but these attempts have shown the difficulties of merging metrics with different numerical ranges and even physical units. All these studies have raised the need of a deeper knowledge in order to determine independent grasp quality metrics which enable a global assessment of a grasp, and a way to combine them. This paper presents an exhaustive study in order to provide numerical evidence for these issues. Ten quality metrics are used to evaluate a set of grasps planned by a simulator for 7 different robot hands over a set of 126 object models. Three statistical analysis, namely, variability, correlation and sensitivity, are performed over this extensive database. Results and graphs presented allow to set practical thresholds for each quality metric, select independent metrics, and determine the robustness of each metric,providing a reliability indicator under pose uncertainty. The results from this paper are intended to serve as guidance for practical use of quality metrics by researchers on grasp planning algorithms

A Grasping-centered Analysis for Cloth Manipulation

Compliant and soft hands have gained a lot of attention in the past decade because of their ability to adapt to the shape of the objects, increasing their effectiveness for grasping. However, when it comes to grasping highly flexible objects such as textiles, we face the dual problem: it is the object that will adapt to the shape of the hand or gripper. In this context, the classic grasp analysis or grasping taxonomies are not suitable for describing textile objects grasps. This work proposes a novel definition of textile object grasps that abstracts from the robotic embodiment or hand shape and recovers concepts from the early neuroscience literature on hand prehension skills. This framework enables us to identify what grasps have been used in literature until now to perform robotic cloth manipulation, and allows for a precise definition of all the tasks that have been tackled in terms of manipulation primitives based on regrasps. In addition, we also review what grippers have been used. Our analysis shows how the vast majority of cloth manipulations have relied only on one type of grasp, and at the same time we identify several tasks that need more variety of grasp types to be executed successfully. Our framework is generic, provides a classification of cloth manipulation primitives and can inspire gripper design and benchmark construction for cloth manipulation.Comment: 13 pages, 4 figures, 4 tables. Accepted for publication at IEEE Transactions on Robotic

Whole-Hand Robotic Manipulation with Rolling, Sliding, and Caging

Traditional manipulation planning and modeling relies on strong assumptions about contact. Specifically, it is common to assume that contacts are fixed and do not slide. This assumption ensures that objects are stably grasped during every step of the manipulation, to avoid ejection. However, this assumption limits achievable manipulation to the feasible motion of the closed-loop kinematic chains formed by the object and fingers. To improve manipulation capability, it has been shown that relaxing contact constraints and allowing sliding can enhance dexterity. But in order to safely manipulate with shifting contacts, other safeguards must be used to protect against ejection. “Caging manipulation,” in which the object is geometrically trapped by the fingers, can be employed to guarantee that an object never leaves the hand, regardless of constantly changing contact conditions. Mechanical compliance and underactuated joint coupling, or carefully chosen design parameters, can be used to passively create a caging grasp – protecting against accidental ejection – while simultaneously manipulating with all parts of the hand. And with passive ejection avoidance, hand control schemes can be made very simple, while still accomplishing manipulation. In place of complex control, better design can be used to improve manipulation capability—by making smart choices about parameters such as phalanx length, joint stiffness, joint coupling schemes, finger frictional properties, and actuator mode of operation. I will present an approach for modeling fully actuated and underactuated whole-hand-manipulation with shifting contacts, show results demonstrating the relationship between design parameters and manipulation metrics, and show how this can produce highly dexterous manipulators

Predicting grasp success in the real world - A study of quality metrics and human assessment

Grasp quality metrics aim at quantifying different aspects of a grasp configuration between a specific robot hand and object. They produce a numerical value that allows to rank grasp configurations and optimize based on them. Grasp quality metrics are a key part of most analytical grasp-planning approaches. Additionally, they are often used to generate ground-truth labels for synthetically generated grasp exemplars required for learning-based approaches. Recent studies have highlighted the limitations of grasp quality metrics when used to predict the outcome of a grasp execution on a real robot. In this paper, we systematically study how well seven commonly-used grasp quality metrics perform in the real world. To this end, we generated two datasets of grasp candidates in simulation, each one for a different robotic system. The quality of these synthetic grasp candidates is quantified by the aforementioned metrics. For validation, we developed an experimental procedure to accurately replicate grasp candidates on two real robotic systems and to evaluate the performance of each grasp. Given the resulting datasets, we trained different classifiers to predict grasp success using only grasp quality metrics as input. Our results show that combinations of quality metrics can achieve up to a 85% classification accuracy for real grasps

Autonomous Robotic Grasping in Unstructured Environments

A crucial problem in robotics is interacting with known or novel objects in unstructured environments. While the convergence of a multitude of research advances is required to address this problem, our goal is to describe a framework that employs the robot\u27s visual perception to identify and execute an appropriate grasp to pick and place novel objects. Analytical approaches explore for solutions through kinematic and dynamic formulations. On the other hand, data-driven methods retrieve grasps according to their prior knowledge of either the target object, human experience, or through information obtained from acquired data. In this dissertation, we propose a framework based on the supporting principle that potential contacting regions for a stable grasp can be found by searching for (i) sharp discontinuities and (ii) regions of locally maximal principal curvature in the depth map. In addition to suggestions from empirical evidence, we discuss this principle by applying the concept of force-closure and wrench convexes. The key point is that no prior knowledge of objects is utilized in the grasp planning process; however, the obtained results show that the approach is capable to deal successfully with objects of different shapes and sizes. We believe that the proposed work is novel because the description of the visible portion of objects by the aforementioned edges appearing in the depth map facilitates the process of grasp set-point extraction in the same way as image processing methods with the focus on small-size 2D image areas rather than clustering and analyzing huge sets of 3D point-cloud coordinates. In fact, this approach dismisses reconstruction of objects. These features result in low computational costs and make it possible to run the proposed algorithm in real-time. Finally, the performance of the approach is successfully validated by applying it to the scenes with both single and multiple objects, in both simulation and real-world experiment setups