FRoGGeR: Fast Robust Grasp Generation via the Min-Weight Metric

Many approaches to grasp synthesis optimize analytic quality metrics that measure grasp robustness based on finger placements and local surface geometry. However, generating feasible dexterous grasps by optimizing these metrics is slow, often taking minutes. To address this issue, this paper presents FRoGGeR: a method that quickly generates robust precision grasps using the min-weight metric, a novel, almost-everywhere differentiable approximation of the classical epsilon grasp metric. The min-weight metric is simple and interpretable, provides a reasonable measure of grasp robustness, and admits numerically efficient gradients for smooth optimization. We leverage these properties to rapidly synthesize collision-free robust grasps - typically in less than a second. FRoGGeR can refine the candidate grasps generated by other methods (heuristic, data-driven, etc.) and is compatible with many object representations (SDFs, meshes, etc.). We study FRoGGeR's performance on over 40 objects drawn from the YCB dataset, outperforming a competitive baseline in computation time, feasibility rate of grasp synthesis, and picking success in simulation. We conclude that FRoGGeR is fast: it has a median synthesis time of 0.834s over hundreds of experiments.Comment: Accepted at IROS 2023. The arXiv version contains the appendix, which does not appear in the conference versio

Grasp plannind under task-specific contact constraints

Several aspects have to be addressed before realizing the dream of a robotic hand-arm system with human-like capabilities, ranging from the consolidation of a proper mechatronic design, to the development of precise, lightweight sensors and actuators, to the efficient planning and control of the articular forces and motions required for interaction with the environment. This thesis provides solution algorithms for a main problem within the latter aspect, known as the {\em grasp planning} problem: Given a robotic system formed by a multifinger hand attached to an arm, and an object to be grasped, both with a known geometry and location in 3-space, determine how the hand-arm system should be moved without colliding with itself or with the environment, in order to firmly grasp the object in a suitable way. Central to our algorithms is the explicit consideration of a given set of hand-object contact constraints to be satisfied in the final grasp configuration, imposed by the particular manipulation task to be performed with the object. This is a distinguishing feature from other grasp planning algorithms given in the literature, where a means of ensuring precise hand-object contact locations in the resulting grasp is usually not provided. These conventional algorithms are fast, and nicely suited for planning grasps for pick-an-place operations with the object, but not for planning grasps required for a specific manipulation of the object, like those necessary for holding a pen, a pair of scissors, or a jeweler's screwdriver, for instance, when writing, cutting a paper, or turning a screw, respectively. To be able to generate such highly-selective grasps, we assume that a number of surface regions on the hand are to be placed in contact with a number of corresponding regions on the object, and enforce the fulfilment of such constraints on the obtained solutions from the very beginning, in addition to the usual constraints of grasp restrainability, manipulability and collision avoidance. The proposed algorithms can be applied to robotic hands of arbitrary structure, possibly considering compliance in the joints and the contacts if desired, and they can accommodate general patch-patch contact constraints, instead of more restrictive contact types occasionally considered in the literature. It is worth noting, also, that while common force-closure or manipulability indices are used to asses the quality of grasps, no particular assumption is made on the mathematical properties of the quality index to be used, so that any quality criterion can be accommodated in principle. The algorithms have been tested and validated on numerous situations involving real mechanical hands and typical objects, and find applications in classical or emerging contexts like service robotics, telemedicine, space exploration, prosthetics, manipulation in hazardous environments, or human-robot interaction in general

Parallel Manipulators

In recent years, parallel kinematics mechanisms have attracted a lot of attention from the academic and industrial communities due to potential applications not only as robot manipulators but also as machine tools. Generally, the criteria used to compare the performance of traditional serial robots and parallel robots are the workspace, the ratio between the payload and the robot mass, accuracy, and dynamic behaviour. In addition to the reduced coupling effect between joints, parallel robots bring the benefits of much higher payload-robot mass ratios, superior accuracy and greater stiffness; qualities which lead to better dynamic performance. The main drawback with parallel robots is the relatively small workspace. A great deal of research on parallel robots has been carried out worldwide, and a large number of parallel mechanism systems have been built for various applications, such as remote handling, machine tools, medical robots, simulators, micro-robots, and humanoid robots. This book opens a window to exceptional research and development work on parallel mechanisms contributed by authors from around the world. Through this window the reader can get a good view of current parallel robot research and applications

Rough-terrain mobile robot planning and control with application to planetary exploration

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2001.Includes bibliographical references (leaves 119-130).Future planetary exploration missions will require mobile robots to perform difficult tasks in highly challenging terrain, with limited human supervision. Current motion planning and control algorithms are not well suited to rough-terrain mobility, since they generally do not consider the physical characteristics of the rover and its environment. Failure to understand these characteristics could lead to rover entrapment and mission failure. In this thesis, methods are presented for improved rough-terrain mobile robot mobility, which exploit fundamental physical models of the rover and terrain. Wheel-terrain interaction has been shown to be critical to rough terrain mobility. A wheel-terrain interaction model is presented, and a method for on-line estimation of important model parameters is proposed. The local terrain profile also strongly influences robot mobility. A method for on-line estimation of wheel-terrain contact angles is presented. Simulation and experimental results show that wheel-terrain model parameters and contact angles can be estimated on-line with good accuracy. Two rough-terrain planning algorithms are introduced. First, a motion planning algorithm is presented that is computationally efficient and considers uncertainty in rover sensing and localization. Next, an algorithm for geometrically reconfiguring the rover kinematic structure to optimize tipover stability margin is presented. Both methods utilize models developed earlier in the thesis.(cont.) Simulation and experimental results on the Jet Propulsion Laboratory Sample Return Rover show that the algorithms allow highly stable, semi-autonomous mobility in rough terrain. Finally, a rough-terrain control algorithm is presented that exploits the actuator redundancy found in multi-wheeled mobile robots to improve ground traction and reduce power consumption. The algorithm uses models developed earlier in the thesis. Simulation and experimental results show that the algorithm leads to improved wheel thrust and thus increased mobility in rough terrain.by Karl David Iagnemma.Ph.D

Deep Learning Approaches to Grasp Synthesis: A Review

Grasping is the process of picking up an object by applying forces and torques at a set of contacts. Recent advances in deep learning methods have allowed rapid progress in robotic object grasping. In this systematic review, we surveyed the publications over the last decade, with a particular interest in grasping an object using all six degrees of freedom of the end-effector pose. Our review found four common methodologies for robotic grasping: sampling-based approaches, direct regression, reinforcement learning, and exemplar approaches In addition, we found two “supporting methods” around grasping that use deep learning to support the grasping process, shape approximation, and affordances. We have distilled the publications found in this systematic review (85 papers) into ten key takeaways we consider crucial for future robotic grasping and manipulation research

Recurrent neural networks for force optimization of multi-fingered robotic hands.

Fok Lo Ming.Thesis (M.Phil.)--Chinese University of Hong Kong, 2002.Includes bibliographical references (leaves 133-135).Abstracts in English and Chinese.Chapter 1. --- Introduction --- p.1Chapter 1.1 --- Multi-fingered Robotic Hands --- p.1Chapter 1.2 --- Grasping Force Optimization --- p.2Chapter 1.3 --- Neural Networks --- p.6Chapter 1.4 --- Previous Work for Grasping Force Optimization --- p.9Chapter 1.5 --- Contributions of this work --- p.10Chapter 1.6 --- Organization of this thesis --- p.12Chapter 2. --- Problem Formulations --- p.13Chapter 2.1 --- Grasping Force Optimization without Joint Torque Limits --- p.14Chapter 2.1.1 --- Linearized Friction Cone Approach --- p.15Chapter i. --- Linear Formulation --- p.17Chapter ii. --- Quadratic Formulation --- p.18Chapter 2.1.2 --- Nonlinear Friction Cone as Positive Semidefinite Matrix --- p.19Chapter 2.1.3 --- Constrained Optimization with Nonlinear Inequality Constraint --- p.20Chapter 2.2 --- Grasping Force Optimization with Joint Torque Limits --- p.21Chapter 2.2.1 --- Linearized Friction Cone Approach --- p.23Chapter 2.2.2 --- Constrained Optimization with Nonlinear Inequality Constraint --- p.23Chapter 2.3 --- Grasping Force Optimization with Time-varying External Wrench --- p.24Chapter 2.3.1 --- Linearized Friction Cone Approach --- p.25Chapter 2.3.2 --- Nonlinear Friction Cone as Positive Semidefinite Matrix --- p.25Chapter 2.3.3 --- Constrained Optimization with Nonlinear Inequality Constraint --- p.26Chapter 3. --- Recurrent Neural Network Models --- p.27Chapter 3.1 --- Networks for Grasping Force Optimization without Joint Torque LimitsChapter 3.1.1 --- The Primal-dual Network for Linear Programming --- p.29Chapter 3.1.2 --- The Deterministic Annealing Network for Linear Programming --- p.32Chapter 3.1.3 --- The Primal-dual Network for Quadratic Programming --- p.34Chapter 3.1.4 --- The Dual Network --- p.35Chapter 3.1.5 --- The Deterministic Annealing Network --- p.39Chapter 3.1.6 --- The Novel Network --- p.41Chapter 3.2 --- Networks for Grasping Force Optimization with Joint Torque LimitsChapter 3.2.1 --- The Dual Network --- p.43Chapter 3.2.2 --- The Novel Network --- p.45Chapter 3.3 --- Networks for Grasping Force Optimization with Time-varying External WrenchChapter 3.3.1 --- The Primal-dual Network for Quadratic Programming --- p.48Chapter 3.3.2 --- The Deterministic Annealing Network --- p.50Chapter 3.3.3 --- The Novel Network --- p.52Chapter 4. --- Simulation Results --- p.54Chapter 4.1 --- Three-finger Grasping Example of Grasping Force Optimization without Joint Torque Limits --- p.54Chapter 4.1.1 --- The Primal-dual Network for Linear Programming --- p.57Chapter 4.1.2 --- The Deterministic Annealing Network for Linear Programming --- p.59Chapter 4.1.3 --- The Primal-dual Network for Quadratic Programming --- p.61Chapter 4.1.4 --- The Dual Network --- p.63Chapter 4.1.5 --- The Deterministic Annealing Network --- p.65Chapter 4.1.6 --- The Novel Network --- p.57Chapter 4.1.7 --- Network Complexity Analysis --- p.59Chapter 4.2 --- Four-finger Grasping Example of Grasping Force Optimization without Joint Torque Limits --- p.73Chapter 4.2.1 --- The Primal-dual Network for Linear Programming --- p.75Chapter 4.2.2 --- The Deterministic Annealing Network for Linear Programming --- p.77Chapter 4.2.3 --- The Primal-dual Network for Quadratic Programming --- p.79Chapter 4.2.4 --- The Dual Network --- p.81Chapter 4.2.5 --- The Deterministic Annealing Network --- p.83Chapter 4.2.6 --- The Novel Network --- p.85Chapter 4.2.7 --- Network Complexity Analysis --- p.87Chapter 4.3 --- Three-finger Grasping Example of Grasping Force Optimization with Joint Torque Limits --- p.90Chapter 4.3.1 --- The Dual Network --- p.93Chapter 4.3.2 --- The Novel Network --- p.95Chapter 4.3.3 --- Network Complexity Analysis --- p.97Chapter 4.4 --- Three-finger Grasping Example of Grasping Force Optimization with Time-varying External Wrench --- p.99Chapter 4.4.1 --- The Primal-dual Network for Quadratic Programming --- p.101Chapter 4.4.2 --- The Deterministic Annealing Network --- p.103Chapter 4.4.3 --- The Novel Network --- p.105Chapter 4.4.4 --- Network Complexity Analysis --- p.107Chapter 4.5 --- Four-finger Grasping Example of Grasping Force Optimization with Time-varying External Wrench --- p.109Chapter 4.5.1 --- The Primal-dual Network for Quadratic Programming --- p.111Chapter 4.5.2 --- The Deterministic Annealing Network --- p.113Chapter 4.5.3 --- The Novel Network --- p.115Chapter 5.5.4 --- Network Complexity Analysis --- p.117Chapter 4.6 --- Four-finger Grasping Example of Grasping Force Optimization with Nonlinear Velocity Variation --- p.119Chapter 4.5.1 --- The Primal-dual Network for Quadratic Programming --- p.121Chapter 4.5.2 --- The Deterministic Annealing Network --- p.123Chapter 4.5.3 --- The Novel Network --- p.125Chapter 5.5.4 --- Network Complexity Analysis --- p.127Chapter 5. --- Conclusions and Future Work --- p.129Publications --- p.132Bibliography --- p.133Appendix --- p.13

Use of Slip Prediction for Learning Grasp-Stability Policies in Robotic-Grasp Simulation

The purpose of prosthetic hands is to restore a portion of dexterity lost through upper limb amputation. However, a key capability of human grasping that is missing from most currently available prosthetic hands is the ability to adapt grasp forces in response to slip or disturbances without visual information. Current prosthetic hands do not have the integrated tactile sensors or control policies to support adaptive grasp stabilization or manipulation. Research on slip detection and classification has been providing a pathway towards integrating tactile sensors on robotic and prosthetic hands; however, current literature focuses on specific sensors and simple graspers. Policies that use slip prediction to adapt grasp forces are still largely unexplored. Rigid-body simulations have recently emerged as a useful tool for training control policies due to improvements in machine learning techniques. Simulations allow large amounts of interactive data to be generated for training. However, since simulations only approximate reality, policies trained in simulation may not be transferable to physical systems. Several grasp policies with impressive dexterity have been trained in simulation and transferred successfully to physical systems. However, these grasp policies used visual data as policy inputs instead of tactile data. This research investigates if rigid-body simulations can use slip prediction as the primary input for training grasp stabilization policies. Since current slip detection and prediction literature is based on specific tactile sensors and grasper setups, testing slip-reactive grasp policies is difficult, especially with an anthropomorphic hand. As an alternative to implementing a system-specific policy, real human grasp poses and motion-trajectories were used to test if the trained policy could replicate known human grasp stability. To acquire the human grasp data, grasp and motion trajectories from a human motion-capture dataset were adapted into a simulation. Since motion-capture only includes grasp and object pose data, grasp forces had to be inferred through a combination of analytical and iterative methods. Simulation contacts are also just approximate models; therefore, slip in the simulation was characterized for detection and prediction. The stability of the converted grasps was tested by simulating the grasp manipulation episodes with no control policy. Viable grasps were expected to maintain stability until the manipulation trajectory caused grasp degradation or loss. The initial grasps maintained stability for an average of 27.7% of the grasp episode durations, though with a wide standard deviation of 35%. The large standard deviation is due to episodes with high hand acceleration trajectories, as well as grasp objects with varying grasping difficulty. Policy training using the imported grasps and trajectories was performed using reinforcement learning, specifically proximal-policy optimization. Policies were trained with and without slip prediction inputs, using different reward functions: a reward proportional to the duration of grasp stability, and a reward that also added a grasp-force magnitude penalty. A multi-layer perceptron was used as the policy function approximator. The policies without slip-prediction inputs did not converge, while the policy with slip inputs and the grasp-force penalty-reward function converged on a poorly performing policy. On average, episodes tested with the policy that used a grasp-force-penalty had a 0.11 s reduction in grasp stability duration compared to the initial grasp duration results. However, episodes that did have improved stability under the learned policy improved on average by 0.38 s, significantly higher than the average stability loss. Moreover, the change in stability duration under the trained policy negatively correlated with the initial stability duration (Pearson -0.69, p-value 9.79e-11). These results suggest that slip predictions contribute to learned grasp policies, and that reward shaping is critical to the grasp-stability task. Ultimately, the trained policies did not perform better than the baseline no-policy grasp stability, suggesting that the slip predictions were not sufficient to train reasonable grasp policies in simulation