Sequential Trajectory Re-planning with Tactile Information Gain for Dexterous Grasping under Object-pose Uncertainty

Abstract — Dexterous grasping of objects with uncertain pose is a hard unsolved problem in robotics. This paper solves this problem using information gain re-planning. First we show how tactile information, acquired during a failed attempt to grasp an object can be used to refine the estimate of that object’s pose. Second, we show how this information can be used to replan new reach to grasp trajectories for successive grasp attempts. Finally we show how reach-to-grasp trajectories can be modified, so that they maximise the expected tactile information gain, while simultaneously delivering the hand to the grasp configuration that is most likely to succeed. Our main novel outcome is thus to enable tactile information gain planning for Dexterous, high degree of freedom (DoFs) manipulators. We achieve this using a combination of information gain planning, hierarchical probabilistic roadmap planning, and belief updating from tactile sensors for objects with non-Gaussian pose uncertainty in 6 dimensions. The method is demonstrated in trials with simulated robots. Sequential replanning is shown to achieve a greater success rate than single grasp attempts, and trajectories that maximise information gain require fewer re-planning iterations than conventional planning methods before a grasp is achieved. I

Active vision for dexterous grasping of novel objects

How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. By dexterous grasping we simply mean grasping by any hand with more than two fingers, such that the robot has some choice about where to place each finger. Such grasps typically fail in one of two ways, either unmodeled objects in the scene cause collisions or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anticipated actions. Our approach has three stages. First, we take a single view and generate candidate grasps from the resulting partial object reconstruction. Second, we drive the active vision approach to maximise surface reconstruction quality around the planned contact points. During this phase, the anticipated grasp is continually refined. Third, we direct gaze to improve the safety of the planned reach to grasp trajectory. We show, on a dexterous manipulator with a camera on the wrist, that our approach (80.4% success rate) outperforms a randomised algorithm (64.3% success rate).Comment: IROS 2016. Supplementary video: https://youtu.be/uBSOO6tMzw

Planning simultaneous perception and manipulation

This thesis is concerned with deriving planning algorithms for robot manipulators. Manipulation has two effects, the robot has a physical effect on the object, and it also acquires information about the object. This thesis presents algorithms that treat both problems. First, I present an extension of the well-known piano mover’s problem where a robot pushing an object must plan its movements as well as those of the object. This requires simultaneous planning in the joint space of the robot and the configuration space of the object, in contrast to the original problem which only requires planning in the latter space. The effects of a robot action on the object configuration are determined by the non-invertible rigid body mechanics. To solve this a two-level planner is presented that coordinates planning in each space. Second, I consider planning under uncertainty and in particular planning for information effects. I consider the case where a robot has to reach and grasp an object under pose uncertainty caused by shape incompleteness. The main novel outcome is to enable tactile information gain planning for a dexterous, highdegree of freedom manipulator with non- Gaussian pose uncertainty. The method is demonstrated in trials with both simulated and real robots

Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects

This paper addresses the problem of simultaneously exploring an unknown object to model its shape, using tactile sensors on robotic fingers, while also improving finger placement to optimise grasp stability. In many situations, a robot will have only a partial camera view of the near side of an observed object, for which the far side remains occluded. We show how an initial grasp attempt, based on an initial guess of the overall object shape, yields tactile glances of the far side of the object which enable the shape estimate and consequently the successive grasps to be improved. We propose a grasp exploration approach using a probabilistic representation of shape, based on Gaussian Process Implicit Surfaces. This representation enables initial partial vision data to be augmented with additional data from successive tactile glances. This is combined with a probabilistic estimate of grasp quality to refine grasp configurations. When choosing the next set of finger placements, a bi-objective optimisation method is used to mutually maximise grasp quality and improve shape representation during successive grasp attempts. Experimental results show that the proposed approach yields stable grasp configurations more efficiently than a baseline method, while also yielding improved shape estimate of the grasped object.Comment: IEEE Robotics and Automation Letters. Preprint Version. Accepted February, 202

Sequential Re-planning for Dextrous Grasping Under Object-pose Uncertainty

Abstract — This work shows how successive grasp attempts can be re-planned to make use of tactile information acquired during previous grasp attempts. Our main contributions are to enable planning of dexterous grasping for high degree of freedom manipulators, and belief updating from tactile sensors in 6 dimensional space. The method is demonstrated in trials with simulated robots. Sequential re-planning is shown to achieve a greater success rate than single grasp attempts, and trajectories that maximise information gain require less replanning iterations than conventional trajectories before a grasp is achieved. I

A Reactive Planning Framework for Dexterous Robotic Manipulation

This thesis investigates a reactive motion planning and controller framework that enables robots to manipulate objects dexterously. We develop a robotic platform that can quickly and reliably replan actions based on sensed information. Robotic manipulation is subject to noise due to uncertainty in frictional contact information, and reactivity is key for robustness. The planning framework has been designed with generality in mind and naturally extends to a variety of robotic tasks, manipulators and sensors. This design is validated experimentally on an ABB IRB 14000 dual-arm industrial collaborative robot. In this research, we are interested in dexterous robot manipulation, where the key technology is to move an object from an initial location to a desired configuration. The robot makes use of a high resolution tactile sensor to monitor the progress of the task and drive the reactive behavior of the robot to counter mistakes or unaccounted environment conditions. The motion planning framework is integrated with a task planner that dictates the high-level manipulation behavior of the robot, as well as a low-level controller, that adapts robot motions based on measured tactile signaOutgoin