Visuo-Haptic Grasping of Unknown Objects through Exploration and Learning on Humanoid Robots

Die vorliegende Arbeit befasst sich mit dem Greifen unbekannter Objekte durch humanoide Roboter. Dazu werden visuelle Informationen mit haptischer Exploration kombiniert, um Greifhypothesen zu erzeugen. Basierend auf simulierten Trainingsdaten wird außerdem eine Greifmetrik gelernt, welche die Erfolgswahrscheinlichkeit der Greifhypothesen bewertet und die mit der größten geschätzten Erfolgswahrscheinlichkeit auswählt. Diese wird verwendet, um Objekte mit Hilfe einer reaktiven Kontrollstrategie zu greifen. Die zwei Kernbeiträge der Arbeit sind zum einen die haptische Exploration von unbekannten Objekten und zum anderen das Greifen von unbekannten Objekten mit Hilfe einer neuartigen datengetriebenen Greifmetrik

Active haptic perception in robots: a review

In the past few years a new scenario for robot-based applications has emerged. Service and mobile robots have opened new market niches. Also, new frameworks for shop-floor robot applications have been developed. In all these contexts, robots are requested to perform tasks within open-ended conditions, possibly dynamically varying. These new requirements ask also for a change of paradigm in the design of robots: on-line and safe feedback motion control becomes the core of modern robot systems. Future robots will learn autonomously, interact safely and possess qualities like self-maintenance. Attaining these features would have been relatively easy if a complete model of the environment was available, and if the robot actuators could execute motion commands perfectly relative to this model. Unfortunately, a complete world model is not available and robots have to plan and execute the tasks in the presence of environmental uncertainties which makes sensing an important component of new generation robots. For this reason, today\u2019s new generation robots are equipped with more and more sensing components, and consequently they are ready to actively deal with the high complexity of the real world. Complex sensorimotor tasks such as exploration require coordination between the motor system and the sensory feedback. For robot control purposes, sensory feedback should be adequately organized in terms of relevant features and the associated data representation. In this paper, we propose an overall functional picture linking sensing to action in closed-loop sensorimotor control of robots for touch (hands, fingers). Basic qualities of haptic perception in humans inspire the models and categories comprising the proposed classification. The objective is to provide a reasoned, principled perspective on the connections between different taxonomies used in the Robotics and human haptic literature. The specific case of active exploration is chosen to ground interesting use cases. Two reasons motivate this choice. First, in the literature on haptics, exploration has been treated only to a limited extent compared to grasping and manipulation. Second, exploration involves specific robot behaviors that exploit distributed and heterogeneous sensory data

Enhanced Learning Strategies for Tactile Shape Estimation and Grasp Planning of Unknown Objects

Grasping is one of the key capabilities for a robot operating and interacting with humans in a real environment. The conventional approaches require accurate information on both object shape and robotic system modeling. The performance, therefore, can be easily influenced by any noise sensor data or modeling errors. Moreover, identifying the shape of an unknown object under some vision-denied conditions is still a challenging problem in the robotics eld. To address this issue, this thesis investigates the estimation of unknown object shape using tactile exploration and the task-oriented grasp planning for a novel object using enhanced learning techniques. In order to rapidly estimate the shape of an unknown object, this thesis presents a novel multi- fidelity-based optimal sampling method which attempts to improve the existing shape estimation via tactile exploration. Gaussian process regression is used for implicit surface modeling with sequential sampling strategy. The main objective is to make the process of sample point selection more efficient and systematic such that the unknown shape can be estimated fast and accurately with highly limited sample points (e.g., less than 1% of number of data set for the true shape). Specifically, we propose to select the next best sample point based on two optimization criteria: 1) the mutual information (MI) for uncertainty reduction, and 2) the local curvature for fidelity enhancement. The combination of these two objectives leads to an optimal sampling process that balances between the exploration of the whole shape and the exploitation of the local area where the higher fidelity (or more sampling) is required. Simulation and experimental results successfully demonstrate the advantage of the proposed method in terms of estimation speed and accuracy over the conventional one, which allows us to reconstruct recognizable 3D shapes using only around optimally selected 0.4% of the original data set. With the available object shape, this thesis also introduces a knowledge-based approach to quickly generate a task-oriented grasp for a novel object. A comprehensive training dataset which consists of specific tasks and geometrical and physical knowledge of grasping is built up from physical experiment. To analyze and e fficiently utilize the training data, a multi-step clustering algorithm is developed based on a self-organizing map. A number of representative grasps are then selected from the entire training dataset and used to generate a suitable grasp for a novel object. The number of representative grasps is automatically determined using the proposed auto-growing method. In addition, to improve the accuracy and efficiency of the proposed clustering algorithm, we also develop a novel method to localize the initial centroids while capturing the outliers. The results of simulation illustrate that the proposed initialization method and the auto-growing method outperform some conventional approaches in terms of accuracy and efficiency. Furthermore, the proposed knowledge-based grasp planning is also validated on a real robot. The results demonstrate the effectiveness of this approach to generate task-oriented grasps for novel objects

Tactile mapping of harsh, constrained environments, with an application to oil wells

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. [110]-114).This work develops a practical approach to explore rough environments when time is critical. The harsh environmental conditions prevent the use of range, force/torque or tactile sensors. A representative case is the mapping of oil wells. In these conditions, tactile exploration is appealing. In this work, the environment is mapped tactilely, by a manipulator whose only sensors are joint encoders. The robot autonomously explores the environment collecting few, sparse tactile data and monitoring its free movements. These data are used to create a model of the surface in real time and to choose the robot's movements to reduce the mapping time. First, the approach is described and its feasibility demonstrated. Real-time impedance control allows a robust robot movement and the detection of the surface using a manipulator mounting only position sensors. A representation based on geometric primitives describes the surface using the few, sparse data available. The robustness of the method is tested against surface roughness and different surrounding fluids. Joint backlash strongly affect the robot's precision, and it is inevitable because of the thermal expansion in the joints. Here, a new strategy is developed to compensate for backlash positioning errors, by simultaneously identifying the surface and the backlash values. Second, an exploration strategy to map a constraining environment with a manipulator is developed. To maximize the use of the acquired data, this work proposes a hybrid approach involving both workspace and configuration space. The amount of knowledge of the environment is evaluated with an approach based on information theory, and the robot's movements are chosen to maximize the expected increase of such knowledge. Since the robot only possesses position sensors, the location along the robot where contact with the surface occurs cannot be determined with certainty. Thus a new approach is developed, that evaluates the probability of contact with specific parts of the robot and classifies and uses the data according to the different types of contact. This work is validated with simulations and experiments with a prototype manipulator specifically designed for this application.by Francesco Mazzini.Ph.D